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ABSTRACT | a he - 7 es me 
»* - This ‘conferend focused on two themes: research in Me 
testing and the cooperative rewearch program, and testing itn the 7 
language arts. The asorfiénq ses#ion was concerned. with, the Lapact of 
the federal’ cae pores) a recesveh Program on educaticnal research. oe 
Papers were erftitled: The Suppert of. Nea sureRent Projects: by the ‘“ 4 
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* SPHERE were two thorries upon which the 1960 Invitational Cor- 


ference on Testing Problems focused its attention: “Research - ; 
, in Testing and the Cooperative Research Program” and “Test: 


“ins in'the Language -Arts.”.Tho speakers covered a _widg range 
“of Interests within both these areas, 
cussion of them ‘Yhat followed: provided a program that was 
pertingnt and rovocative—valuable’ to those who specialize 


in the problémis:of measurernent and stimulating to all who 


- are concerned about the problems of education in general. 


ference Chairman, was, responsible. for assembling the pro-” 
* gram and speakers, deserves our praise and prelate: ‘I 
would also like to express: our. thanks to the \distingujshed 


spCakers whose papers'we are peer to et ns edition 


, of the proceagians vO 


= Henty Ctlauncey 


‘Thetr papers and the dis-- 
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Dr. John B. Carroll of Harvatd University who, as’ Con- y 


«President . 
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TWo IDwAs' entered my mind as L cast about for themes around ~~. 
“which to build the program for the 1960 Ipvitational Con- . _ 
ference on :Festing Problems. One cange from the. fact. that . 
theinterest of the Federal Government in educational research 
-- has in the last few years increased: to a point where'it is a 
_Major.sougce of support for programs of research in’ educa- 
tion, including research in edutational tests and measure- | 
ments. The other was my own special interest in problems . 
of ‘testing in the “language arts:” Both these ideas. seemed to 
have wide enough appeal to serve ag themes for the Con- 


ference, andyhey had interesting connections with each other: °. 


it happened, for example, that several developments in 
language ‘testing had been Tmade ‘possible as a result of 
"government, supports , e. 3 % vr 
As it turned out, hs interplay of ideas ‘made fort.a nfest 
interesting and ,Stimulatinig conference, when brought to 
reality: by a ‘panel of speakers who were not only well informed | 
-but also able to make creative ‘and thought-provoking contri- 
, butions. t is a measure of the success of the Invitational 
Conferences that so many people—this year more than 550— 
make the ¢ffort to come to the Hotel Roogevelt to héar the 
speakers: _ person and listén to the sometimes heated dis-’; . 
‘cussion. | dandot believe that many were disappointed: this . 
year. a ae ee —_ an io 
- ‘Tho..morning session was concerned with the. impact of . 


-.',) Just one ‘major prdgrant: 6f Federal support to education 


_ research——the. Cooperative Reseafch Program administastd) 
by tHe Us S. Office of Education. The first paper, read py 


‘Herbert’ S." Gonrad for Roy M. Hall, Guhfortunately absent - -. 


{ 


_ bevause of desert ou ‘a destpiption of thé program if gen-. 
", ral, The remain 
‘ me ‘. te . 7 Vans ap 


ig oo 88 ‘ ‘ ‘ 
ah » i sk Baill . ‘ ack 


ng three papers'were hy the directors. of 8 


‘three ‘out of thé many research projects supported by this 
program: Norman F¥retlerikgen, Educational Testing Service;, 
Robert M.. W, Travers, University ef ‘Utah; and Carson 
McGuire, University of Texas. -- | els Stas 
a Adjourning for lunch, the conferecs had, the pleasure of | eee, 
: hearing Dr. Arthur Adams, President’ of the America Coun- = 4 
cil on Education, brilliantly and entertainingly speak on “The” * 2, 
_ ' Pave of Change” Dr. Adams described some of oe perierices 
as an edacational consultant in the new-Afric! bp ifiehe and, 


ope ery 
s . 
. . > a 
Bay. 7 24 on ® 
: x * + ‘ ra : 
‘ 
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reminded us of the pace_at which changes are occurring in. 
. all phases of education, international ‘as well as domestic, and: - ee 
Of our responsibilities in ketping abreast of them, =. -  °.| = 
Fhe afternoon session turned to the:theme of “Tosting in. * 
the Language Arts.” The speakers were. Harold C..M rin, 0. 
_ Harvard University, Wilmarth: H.. Starr, Modern Lanfaare all 
*» . Association and New York University; and Irving’ Lorge, : 
- Teachers ‘College, Columbia University. <2 a ee 
l-am glelighted to have this opportunity -to‘express my great | 
appreciatidh to the speakers for the thought and effért they 
devoted ‘to the preparation sf their remarks, and for.the excel- - 
lent presentations they made at the Conference. . 
It is also a pleasure to.have ‘my turn at representing the 
. © profession at large, and the Conforence invitees in particular, 
‘in thanking the’ Educational Tésting Servicu—its President, “ 
Henry Chaungey, and all its staff members who contributed ~~” 
w the success of this annual event:—for providing so happy , 
“an occasion at which those’ conceritetwith . problems of 
~ educational measurement can’ meet. together ‘and: exchange ~ 


information and~deas, | a ae o 7a ; — 
: ar | i — John B. Carroll. 7: -* 
. 4% Tae o, a | Chairman * | 
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: OUR THEME this morning is “Research in. Testing; and ‘the . cen 
py COCHRANE Research Program.” It is prompted by the recently =” 
rowing and massive interest in’ educational research” on the: 

art of the Federal Government. To be-sure, the ‘government -’. : 

TS “has been involved ‘in ‘educational research, of a sort, for a 
good number of Years. Group testing recgived its first impetus’ :. 
-.'~ from the Army program of World War'I, and the’ programs 
w 7 oof military personnel researgh engendered by World War Il 
+ are well known to all. ‘But most of this was'only on military’ 
nF problems; ‘any relevance to édweational problems.in the schools’ 
” @ ~ Was incidental. ‘After many yeiirs ‘in’ which’ thé. Federal Goy- °° 

* ernment phid, scant attention to’ educational: research, and 
- testing in*the public schools, the Congress; suddenly had a’ 

change of heart and in 1954 passed Public-Law 631, which 

gave rise to the Coop@rative Research Prograrii, of the United . 

| States Office of Education. = 

It is heartening to note. that this’ program: was not one of 

. the reactions to the. orbiting: of the first Spytnik. Much of" the: 
- impetus “for the program came from the interest of legislators -, 

_ In the problems of rental retardation. Having looked recently - 

" into’ the research, literaturg ori mental retardation, I can attest 

. that there was ‘ample reason to put funds into. this. kind x 
" research, But ghowever costly the.care of mental efectives js 

for our society, these people constitute only perhaps two or, :. 

“three per centlofthe popufation of school-going age; aha 
up a Cooperative Research Program, ‘with more. than half of *. 
‘the funds originally allocated to research’ in, mental, retarda- es 
. tlon-was a case Ot ‘letthng ‘the tail Wwag the dog. But at any 
rate’ let us-be thankful fof the: dg, with or, without tail. ee 

e ee 
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7 oa Thea, caine Sputnik. F agd another wrong Xeasqn (hpwever 
* -, happy, the result) for increasing Federal ;tid* educatioral .. 
research. ‘The outcome was the: National Defe sg Education ° 
-. , ' ‘Aet of: 1958. Here again the liw-oxhipited what many have..: . >. - 
; - a regarded: 9s fy improper, balance among the Various” kinds Tg 
"ff  *of Support that could havé- been giveng‘The-emphasis was... 
> * upon ‘sciehce, mathematics; and foretgm Jdnguages, with ay. 
. substaryial bow in-the direction of school guidance programs ye 
and the development of éducational “media” such, as television;*. 3." 
a’... . ofilims, ‘arid even. teaching machines. By. the quirks of’ the. 
- legislative process, éducational yesearch under the National - 7 
Defense Edutation Act has-been limited to research,in foreign 
_- language teaching “and in the development of cdugational 
\_.- media.‘ Even*research in.the tedching of’ scienc - andignathe: 
: matics’ was excluded unless it could’ ‘bet brought user’ the 
le “+. heading of media résearoy. Nevertheléss,-we can bethankful. —* 


® a 


» ¢. _. ’, for.the many godd thingstprought about by the NDEA,... *. | 
. "At seemed therefore’ fide o-dedicate 4 part of ebir, PIO. 
gran today to a recogn of these Houslshing prograiny of *-. "4 
i cderal support to educatioYi) research “ahd testing.“Through ; 


1, * * Wihat we say here today we, 
_ .' attach @ educational ‘reséare 


. t 
tgs edcydion. People, professionally coneérnéd with educational | °. a 
soe S tesedrch and measurement at} gratified’ to nde the establisH- oo 
P ment of these programs of Federal support. ‘Fhix, conférence es 
aa ‘Provides ‘ah occasion for looking at ‘some of the Orst,.frutes® . + 
of these programs, «2 0-7; aes 


y 
e 


: Ma _. project@on testing thé administrative: abilities of ¢lementary ° 


Ps 


2 ocoag, Stalenewill be stimulating. i ‘ 


‘ : e. on 
' oy _@ 7° 
, an “ 
7 , ‘ |. . q . : 
4 oo ‘ ‘ . Bs a . , 
——_e Ce Te ge ohn B®. Carroll. 


, » “4 Educatidn and a person whose work in:education’l measure: 
ment \ag@ statistics, is familiar to you. Actually, Dr. Oonrad* ’ 
was a co-author, of¢the paper We are i° hear, entitled, “The | 

_ Support of Measurement Projects by the Cooperative Rescarch. 
Program” = ae rs a a. 

p ops towing this, you will “hear three papers, each one by.a:. 

«director of one of the scores of projects which: have been or :. 
are being supported by the Cooperative’ Resfarch Program, | 
was faced with an embarrassment of riches as7I tried to gelect ; 


* cate the reasons for my choices—let ps simply leave it that. 


wy 


interesting and provocative speakers; I shall. not try to; indi- re 


e 


4 


a~ 


I attempted to bring you addresses: which’ would give a satiy-. « ‘! 


factory impression of ‘the diversity and, scope of “tH® r@seatch. 


.« | "”. being supported bythe Coaperative Reseatch Program! ‘ 
i. The first of these papers is “In-Baskets ‘Tests and Factors ~ 


: 
-* y/lit Adniinistrative Performance,” by Dr. Norma Frederiksen’ 
of the Educational, Testing Service, desokjbing results of a, 
» sehool principals. For this: project,. ETS joined forces with 
Columbia Univaesity. ee ee 
The seeond.is “Models of Teacher Behavior infthe Cldsg- -— 
coon,” by Dr. Robert: M. W/T ravers, Chairman of?the Depart. 
ment of -Edycationil Psychology, University of’ Utah, .and_ 
*“. author of several books on measurement and’ educational 
-. research, This paper fs in the nature of a progress report on 4 


His Investigation! of tthe process of tvaching. |. 

_ <The third is “Fhe Prediction of ‘Talented. Behavior inthe — 
- Junior High §chool,”.by Dr, Carson. McGuire, of the. Depart: - 
ment of Educational Psychology at the University of Texas, 
"a psychologit who has long béen interested in ways of meaqur- 
ing thagless accessible aspéets.of human behavior, [ am gure: 
that his approach to the .jncreasingly’ popular “search for 
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4HE UNITED STATES OFKICE pf Education “provides support, . 
through its Cooperative ‘Research Program, for research of. 
" significance to: education. This Program is both extramural" ' 
, and cooperative: “extra-mural,” in that .the-research ariginatas®. —. 

_ with, and ts conducted by, investigators and institutions out-* 


Cooperative Restarch f j 
Program oi the U.' 8. 
Oftioé of: Education 


" Ce} 
‘ 
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rs 


~4 side the Goyernment; and “cod rative,” in that those who - 
NS nk ft ean 


< 


7 


;, 14 niflcance tg education is bligible for consideration: It may be 
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originatd research share;-with the Office ‘of Education a 
> vital interestin the-project and age willing to contribute toward 
" ts" fAnaricing..4 broadly ; cpresentative group of educational 
mescarchers .agsists the. Office.in, evaluauing the proposed 
| Researchy 0 eit ve Pee Pe. 
Publie-Liayy 531, under which the Cooperate Research Pro- 
fram operates, (authorizes the Office of Education to “enter 4 . 
Widen financed, cooperative arrangements ~ — 3, 
with uniyersities ‘and collegés and State'educational agencies. +. 
for the conduct of research, survcys,,and demonstrations in: 
the fletd of tducition.” ‘The: particular dispects ‘of education. = *, 
¢ cligiblo for support under the Gooperative Research Program = 
“Ure not ‘gpecifically delingated: in: tht -law, ‘Any project of sig. |e: 


"| & project concerned with Pre-school,‘ elémentaty, secondary,“ 
“higher, ‘or adylt education, It may’ be. congertied with, public. 
- or private 7 ane It may qeal with ‘the teaching-Jearhing » 
sityation, administrative functions, sdcivlogical adpects of the 
classraom atid community, psychological theory, or principles °’ 
of measurement, Te may deal with the means employed br the 
“products achieyed. It may deal with the, past or the present, © 
of, ‘indeed, with the futuro. In essence,. of course, all good .. 
roacaroh, ts future-minded: “thd, pdst.{s prologue,” and by 
definition, the prose {dousn’t last'very long.) 
eo a oan ee Se ere Mee eee i 
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— Research and development in the arca of measurément.deserve 
the interest and support of all educators: partly {because 


\ measurement remdérs immediate practical service, but. more. 
, important, because advances in- measurement frequentay spark 


significant advances: in education, Many ofthe inyestigadons 
supported by the Cooperative Research Program of the Office 
of Education ‘involve the construction, and refinement of 
Hiab phe instruments. Some examples of such invdstiga- 
tions- Aumong our proudest: products-—will be presented by 
the other sPSukers on thiy morning's panel. In the time at 
my disposal, let’me describe to you a variety of other studics 


r measurement. .* Me 
\ ; 
. ' Measurement Projects Supported 
4 oe? the Cooperative Researoh Program 


. 
oF 


voy oe ram sucks to refine existing dustruments (published or 
ty WG RMDished), or to improve our understanding of them, Ah 


Vet the Human Development Clinic of Florida State 
Sultans. Ne. Kennedy is standardizing the Third Revision 

Of stl Mord Binet Intelligence Stalecon Negro elementary 
Lichddl children in the Southeastern United States, The norm- 
yg Hmph will consist ‘of 1,800 Negro students, grades 1 
SroughS, from five States iy the Southeast. In order to assure 
representativencss,*the sample will be stratified by suchr fac- 
‘tors’ as urbanization, socloeconom¥e status, and eslucational 
facilities available, Bidgraphic data will be collected in order 
2 - to comparg the representativeness @f the sample with data 


to perform some validation analyses by ‘correlating Stanford- 
Binet scores-with achievement test scores .and with academic 


item intercorrelations, will also be carried out. This study ts 


., A second type of study supported by the Cooporative Re- 
sourch Program includes the development of new instruments 
~~often ax part of a largor- undertaking in the fleld of measure- 
¥ gonnection, the word “new” may require some 


‘ 4 a hy . 
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‘performance, Item analyses, ¢ncluding fagpor * analysis of , 


under the. Cooperative Research Program ,which deal with 


from the United States Census .Bureau.' Dr. Kennedy plans . 


scheduled for completion ffi abou a year, (December 1962). 


" qualification. Some tests are “newer” than othors;.it is difficult, 
sometimes, to distinguish between inrfovation, r@inement, and 
“a ‘recombination: all thread may, ofcourse, occur together. . : 
cog . Perhaps the best-known of the’ measurement projects; sup- 
: “SS: ported by the Cooperative Research:Program (in conjunctioh 
“Swath other Government agencics) is the so-called “Project 


\, torests, and’ othet characteristics’ ‘of close tos half a million 


high schogls in the United States (both public and private), 
The study is directed by Drs. John C. Flanagan and John 'T. 
Dailey, under the auspices of the University of Pittsburgh. In 
order to assure representativencss of the sample, schools were 
stratificd (with the aid of a molern, high-powered computer ) 
on such factogg as geayriiphic looation, size of the twelfth 


ten, and twenty years after graduation, from high school. 
The major goals of Project Talent, other than the develop- 
ment and norming of the various tost-Instruments, can .be 


very bricfly summarized as follows:  - 
_1. ‘To: provide a sclentifically planned national inventory 
of youth, ; 


2. ‘To determine the specific pattgrns of aptitudes, abilities, 


3. To carry out follow-up studies (o determine the educa- 
ional experiences which best contribute to the dgvelopment, 
of these aptitudes and abilities. 


oo ; ” NO: 
4. To determine the guidance procedures most. effective 


~ him personal satisfaction and success. 


The battery of data-collection instruments was developed 

" —_ espectally for Project Talent. No commercial tests are inclutled 
in the battery: There is a Student Information Blank, a Student 
Interést Inventory, and a Studomt Activity Inventory, plus three 

ir — questionnaires to be completed by schoo) personnel concerning 
. 7) v.the schools themselves, Numerous at alla 


‘ tests were administered to ghe. students, woe 
- -\ i . te a ee 
‘ 9 | ae > ou t . . ; -b Z ‘ . : 7 “ as 
ENC ro atg rr « re eS 
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ee ‘Thi study will survey the aptitudes, abilities, in-— 


spupils of grades 9, 10, 11, and 12 in: appr Aimately 1,000 .. 


grade class, af holding power of the school..Follow-ups of | 
‘studonts in the sample are planned at intervals of one, five, 


and interests which-are best adapted to various colloge courses - 
and curcers. G: ra re 


_{n assisting vach student to seléet a carcor whith will assure 


aa ° . ® - as A 
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The Talent battery of tests is not, and We not 
* avallable commercially. A limited number of 

can be made available, however, for qualifi i 

* Dr. Sarnoff A.,Mednick of the Universit 
developed an associative thtory of creative thinking, gnd 
carried out some preliminary experiments which seem to,sup- 
-  poyt. the theory. An objective test entitled Remote Associates 
» ‘Test, Which is :an oper¥tional statement of/the theory, has 
been devoloped for measuring individual diff¥rences fy creative 

2 3 rs . ‘ ne 


4 


talent. e , ~ 
“Ttems in the Remote. Associates. Test resent thrée words ; 

for which the subject is to respond with fourth word which 

serves as a connective link among thom 

words such as the following arg given: 


surprise li fa birthday ~ % 
The answer is party-—surprise party, party ling, birthday party, 
Another example is ' fo : , 
base ,  sngW dance ; 
‘The correct response 4s ballAbaseball, snowball, ‘'dance-ball. 
Try this one. ; i a” . 
railroad =» / girl, * class : 
The Remote histo Test is béffig administered expert- 
mentally’ in several grdduate schools as a potential selection “ ’. 
device. It appears to/be a promising instrument, carrelating 
In the Jow 30's with tests of verbal ability. Mednick’s work 
has been conduciéd mainly at the college level. He has a 
Cooperative Resotrch gontract to extend the réscarch to the 
secondary levely from this should develop severaf forms of, . + 
the Remote .Adsociates ‘Test for high school students.: a 
Frederick B, Davis and Gerald 8. Lesser of Hunter College 
ure colnvestigators\on two Coopérative Research, projects in as. 
which they have developed the Hunter. Aptitude Scales. for ' 
4 ° , Gifted Children, to be used in identifying pre-schdol arid ele- 
mhontary school children, gifted in one or ‘more of five areas. 
The five scales which were developed are as follows: , a 
. 1, Space Condeptualization, @ cluster of ‘skills in judging. = 
_. wpatial relationships, the' sizes gfsobjects, and movements in # 
“space. , : x a ce a rar 
Fo Vocabulary: dofiried as memory -for word meanings— 
Le 1, ‘ : an’) 


. For example, three 


° 
a 


. 
* 
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-events, and. actions. The 


\ speaking per 


’ —_ 


| | . 
1960 Invitational Conference on Testing Problems ~ 


with reasoning elements, such’ as those required in a verbal 
unalogies test, reduced to the minimum, _ 

3. Number Ability: closely defined. as memory of the funda- 
mental combinations in additign, subtraction, multiplication, 
and division, as well as skill in using these accurately. ’ 

4.. Reasoning: the ability to.formulate, con€epts, to weave 
together ideas and concepts, and’ to draw inferences from 
them. a : 
5. Science: knowledge of sclentiffe principles and informa- 
tion, . — 

These tests are all individually administered, and requite. 
no reading on the part of the examinecs, 7 

Working at the other end of the human-ability scale, Dr. 
Iloyd M. Dunn, Coordinator of Special Education, George 
Peabody College for ‘Koachers, has developed The Peabody 
Picture Vocabularly Test (PPVT). The PPVT, which is being 
distributed by a test, publisher, is designed:to measure. com- 
prehension of the spoken waftd, including the ability to, asso- 
ciate verbal symbols with ptorial-representations of. objécts, 

ost consists of 150 items and has 
two forné. Tt is individ y administered, yielding three types 
of derived scores—-age scores, quotients, and percentiles. 
Althougfy the PPV& was constructed primarily for use with 
mentally fetarded individuals, it can be given to-any English: 
in between 2% and 18 years off age.who is able 
o hear words, see the drawings, and has the ability to indicate 
‘sor nd in.a manner Which communicates, 
\prs, George G. Stern and Joseph M. Masling, Syracuse 
ersity, conducted a Cooperative Research Project entitled, 
“Undonscious Factors in Carcer Motivation for Teaching.” The 
pusgoXe of this study was to develop suitably instruments for 
ussessikg ‘unconscloys factors, In career motivation among 
teachers), andsto refate such factors. to performance in spécific 
areas of the elementary school tédching ‘protess, ‘The test, 
developed with the help of public school administrators, supor- 


visors, and teachers, consists of descriptive sketches of a 


riumber of fhottvatidnal: typeatencountered among teachers, A. 
sample of Individuals considered representative of each type 
was then aclocted, interviewed, and tested. The data obtained: 


‘from these” subjects provided a basis: for building objective , 
‘se ‘ : “ , : ‘ aes 
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- diagnostic instruments: for the idehtiftcatton of ten motivas 


tlagal types. These instruments were admipigtered to samples’ 
of . teacher-trainocs and practice teachers. ¢lassroom rating 


- scales: were also developed and tested in ubservations oO 
rf 


ww 


: elementary school classes. — - 


An additional type of: ‘Cooperative Research Investigation 
falls in the area of application of measuring instruments to 


the educational program, This involves some of the guidance 


and placemen} functions of the schools. One such study, 


“Investigation pf the Use of Statistics in Counseling Students,” - 


was carried oyt by Robert E. Hewes, Registrar, and Richard 
W. Willard, S®itistical Analyst, of the Massachusetts Institute 


: of Technology. They. investigated the potedtial usefulness of 
‘ certain statistics as an aid to M.LT. students faced with 


ry 
t 


' tiveness\ Dr. Stanley fe to analyze c 


choosing their college careers. Profiles-of the students on 
academic measures were compared with profiles of. students 
in various majors. The profile analysis of the ‘academic 


measures added no useful information regarding the students’ © 


predicted likelihood of: graduation in the various majors. A 
student with a p@or prognosis in a mare difficult major had 
little better prognosis in a less dif¥cult major. Ope may infer 
that, among the highly selected students composing the M.LT. 
‘enrollment, diff€rences must be accounted for in other terms, 
such as thos¢ of interest and motivati 


The Cooperative Research Program has also sponsored” 
resdarch investigations concerted. with increasing our ur a 


knowledge of measurement theory. 

Dr, Julian C. Stanley at the Univarsity of Wisconsin ‘is 
‘currently conducting a Cooperative Research Project entitled 
“Development and Analysis of Experimental Designs for Rat-’ 
. ings.” Studies involving ratings are usually. partially invali- 


‘dated by exccsyively high intercorrelations among the traits — 
ters tend to rate indiyiduals too much-in terms’ 


rated; since 
‘of a! 
pletely a fuur- 
dimensional matrix, where cach rater rated each individual . 
more than once on each trait, for sources of bias. Hé hopes. 
ty provide a general analytical basis for. minimizing biases 
‘and, maximizing . independence of traits. Statisticil adjust- 


¢ factor, such as scholastic rac on general attrac- 
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eve 
t. 


1960 tuttatignal Conference on “Testing Probiems 


which not every rater rates every individual on every trait. 

Another Cooperative Research. investigation which may be 
of Interest to you was-a study conducted by Drs. Raymond C. a 
Norris and Howard F. Hjclm. at. George Peabody College for * 
Teachers, ory the effects of ron-normality oe the sampling 
distribution of the product-moment correlation coefficient: It 

i was an empirical | investigation, employing a high-speed 

{ electronic computef. Drs. - Norris: and/ Hjelm used experi- 
| mental populations which were: ana) leptokurtic, and 
skewed, having approximately no correlation and substantial. 
_- ‘correlation. Sampling distributions of the correlation coefficient 
were ‘established for samples of 15, 30,'and 90 cases. In gen- 
cral, the findings were that the effects of non- normality. were 
» not serigus for populations with a true r close to zero;"but were: 
significant for populations having substantial correlation, with 
the nature and direction of the deviation as to. ane \yhe 
_ of hon-normality in the population. 


Mennuvenwari Projecte y . 


{or the Future aT j : 


Te iy clear’ ‘that the’ Coopetative Research Program has 
_ supported’ a Variety of researth investigations in. the neem of 
measurement, We would, however, like to support a sti arger 
variety. Educational measurement is, to ke sure, progréssing: — 
but in. our judgment ‘it is not progressing fast enough, on 
cnough fronts, with enough coordination. 

best that sa Hip ide word, “coordination,” cause alarm, 

Ict me express at once our profound conviction that coordina: 
tion is faroutweighed by inspiration. The kind of coordination. . 
we mean to stress’ is coordination between the experts in: 
.. measurement and the persons. and functions that ‘measure-. 
mont should serve. Experts ténd to develop'a “closed society,” 
reveling {n mutual «appreciation .and in the pursuit of happt- 
“ness through the achievoment of technical perfection—at the 
cost, sometimes, of appreciating others’ needs, Closer liaison 
. ds needed between those who have ideas ‘alsout curriculum, : 

methods, siperviston, schgol construction, school finance, — j 
public relations, delinquency, guidance, amd so on, and those 
who have skills und jdeas in the field of measurement. Too 
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¢ | « child college student; or, rather, not enough has been 

devoted to: otherdreas and. other persons:in the whole field: - 

. of education, cusuremgnt should originate out of, and pro- 
vide tools for, all‘the various‘ aspects of education—from 

I - graduate school down to pre-school, from classroom téaching “+ 

| up to the schoofboard ‘room and the [egislature. This, a3 we __ ae . 

sce It, is the main challenge to educational ‘measurement: = 


much oc measurement has centered on the:school -’ 


today. a ie, oe Se 
There ate some’ other challenges, ,too—all,. as we see It, 

. growing-out of the values, standards, or (if you will) folkways : 

of American_setiqty. For-one thing, we honor the dignity of -. 

man and respect th individual: so the standards of measure,” 

ment are geared, prefyrably, to the individua}Yet education; * 

applied to nearly 50,090,000 persons, must-be“concerned with  - ! 
“ groups; and in sofne delicate domains, where the pressure - - 

for conformity of response may exceéd the impulse toward 

truth, group’ technics of measurement nggy add to validity. 

Another characteristic of our society seems to be“its preoccu-, 

pation .with ‘the present, or, at least, its limited concern for « 

the future--and so installment debt multiplics; overcrdyding , * 


: . 4 ' Per 
of schools is corrected after the event, but not prevent FyOtc. » an 
In the field of measurement, our standards seem to bay fixed a 


largely in the framework of the present. Actually, however,. 
‘ school cHildren grow and develop; what we need ayé-dgvices - 
accurate cngugh to measure not only present status,"but also v 
Change or difference. ,Much more attention-is needed to this ; 
requirement. For thi: purpose, of course, tests far more ac- ,. _ ae: 
‘gurate than those useful for group measurement arc essential; 
- din fact,“we currently have very few tests sufficiently accurate 
‘to Incasure reliably the growth, and development Af individual! - 
school children. A somewhat related problein’ which 1g not =~ 
~ receiving the intensity of research that its iimportance justifies “| 
ls the problem of differential prediction, Here work is needéd 9. 
”. both on the tests: (or other dovicgs) and on the eriteria. Finally, @ 
~ Americans are’ g&nerally accused of belng charmed to excess: 
by considerations of specd, economy, and objectivity-fand so. 
we have the ‘whole field of. measurement dominated by the $s. j¢ | 
paper-and-penvil, machine-scofable format. It’ séems. Fait to 


ask whether the virtues @f speed, economy, and objectivity > 4. 
‘ . . gt 1 . : ; Hd Y my . a . . : | 
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may have been gained at the expense. of innovation, and « 
scope, and-validity. — - , We a ‘ 
.' - * ‘. “The Cooperative-Research Program’is open-to, investigations 
| +4 along all the lines indicatéd above+—as, well, .cf, course, as: 
~2* others. We ,da. not pretend ,to anticipatd the ideas of? able, 
/ invostigators. Ta 8 ag 
Concluding Remarks . + 4 


ra at 


¥ uy 


f; ote - # ae oe 
‘In closing, let me say that the Cooperative: Research Pro-. _ 
* gram of the U. S. Office of, Education is anxious to support ' 
eo as miiny research projetts in educition ys it can, If you are 
| ae planning some research of éignificance to education; ff the: - 
a) research design is sound; if the personnel and facilities are 
adeqte; if the cost fs justifiable; and if you need fipancial |. 
c: ; * . assistance, you should consider ‘Sending your proposed plan | 
@of research to the Cooperative Research Program. We ;: 
' invite and encourage you to subrhit applications for Coopera- - 
— ‘+. tive’ Resdarch funds to conduct’ research, concerned with 
~} °°?" measurement, 
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-. tration, There is a tendency, in evaluating administrators, to. 
-think in terms ‘of one aver-all dimension of goodness or bad- 


of one variabl was too*sjmple. One of our purposes ya to 
try to identify. some’ major dimensions of administrative 


people, - 

"The Study was. a cooperative one, doi by staff members 
«of Teachers College, Columbia Untversity, and Educational 
_ Testing Servicé, When I say we 1 am referring to Dr. Daniel 
_ Griffiths of: Teachers College, Director of the Project; Dr. 


« John“K.*Heniphill of ETS, Associate Director; Dr. Glen Stice, — 


“ Research Agsociate at -ETS, and: myself, The study was sup- - 


“al f r a! In-Basket Tests 
hea { and Factors in 
on Adeninintrative. 


" Re@arch Division, | | : 
Educational Testing ©. °©[ - |- * Performance 


Service - . PP Fh a aes 


THE, PURPOSE of the study which I wif describe this acai 
was to learn ‘nore ‘about performance in educational adminis- | 


behavior; another was to discover how such ‘dimensions might ” 
be related to a variety of other measurable charantertetice of. . 


ness. We felt that formulating the evaluation problem in terms _ 


: ~., ported by g grant to Teachers College: from the Cocparatiye ; fo 


Research Program of the. U. ‘S. Office of Education, | ¢. 


i. on the job 1s:that it is impossible: to’ tell’whether variation in 


ay performance. should: be attributed ta differences. among the . 


_ subjects or. to ‘differericés among the jobs. “In this study we . 
eliminated the ‘job as a source. of variation by: simulating. a 
- school and comnfunity and giting.each one of the 232 sub- 
™ qectettho, job ‘of being’ principal of the same simulated: school., 
_ Enough "information: about this school and community. was’ 


7 '. provided that ‘subjects could reasonably be expectéd to. take 


action on the administrative problems: presented o'them; * 
. The subjects of the inv sna awere '232 ‘elementary - 
: eh pa _ came’ f 


aa . wat agg ai 


Oiié difficulty encountered when one tries to study behavior 


rom meen irate: all over the 


r 
: 4 
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r __additional material for study,‘includigg a Staff "Handbook, the: - 


fe 


“United States, and they constituted squite a varied. group in ‘ 


.terms of. age, experience, and ability, ¢ 


. The test is a fairly elaborate one, requiring five days to. 
~ administer. About 20 people were tested at one sime. The test. 
_ began.on Monday morning, when the examinees were intro- 

duced to the schoal of which each was.to be the rew principal, * 


The simuytated school is Whitman School, located in Jefferson 
in the hypothetical state of Lafayette. Eagh subject was. tem- 
_‘porarily given a new name, Marion Smith. The subjects were 


' instructed that: they were not fo play a, role, that each was — 
to bring to. the new job his: own background of experience, 


his own knowledge‘ qnd* personality.’ During the test he was 
to perform the duties of the new principal of Whitman School. 
The participants spent the first day and a half in learning 


about:Whitman School and its community. First, a film 4trip- 
.. was presented which gave -the participants an‘ over-all view 


of Jefferson. Then the principals were given an opportunity 


- tor study the Jefferson School-C@pmunity Survey which had | 


recently beert completed by the SChool of Education at’Lafay- 


ette State. Next the group.viewed a sound color film. which - ° 


_ took the subjects inside Whitman, where they saw the faculty 


" and children at work. The subjects were given. personnel folders — : 
for the téachers: and staff, a floor plan of ‘the school, and... 


a staff roster.. Study. guides. were provided, to direct the ‘sub- 
. jects’ study of the. materials. 
On Tuesday mornizig the indoctrination was ee er 


. Schook Board Handbook, excerpts from tye Lafayette: school 


- law; copies of the:most recent school census; a class size list," 
_ the school calendar, and_a, report of achievement test scores. © 

* AJl this printed material became the principals’ files through- 

out the week: The’ examinees also listened, to tape recordings — 


of school board oe and Conversations nyolving teachers 
and parents. ° 


At the end’ of the day ‘and a-half of orientation, the subjects vo, 
had as.much information gs would be expected of a new: .. 


_ expect the subjectstg take action on problems arising in thé. 


"principal in an if situation, It was row’ reasonable, to 


eon balan of the school. 
“The. palanee of thé ‘week was Revolt to work sessions fn 


“Norman Federitaen ‘ 
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a ° 84 
. eg : ™ : ry ; . 4 
: oF very sorry to say thst some unaccountable Gelay in shipping 
‘ 11 prevent this year's supply of penoils from resehing your , 
sehool, before September 12, ‘ . 
. : e 


. LJ 
Thies offies egrets to add to your burdened at this busy time of . 
the yeor, but the situation ie, unfortunately, beyond our eontrol, . 
r aaae shat you-cenemake some provisions for coping with thie = <- 
problea. ' ; " y . . 
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Exhibit: I 


_ wor 
: om - oa : . re 
ate Pe a .Norman'Frederikgen ey 
nA : ad | ~ : ra Mla : vay 
tts Awhich each’ Marion Smith performed the duties of theA\grin- 


¥ a - fipal of: Whitman ‘Schqol. All the participants wex¢ presented 
go +, WithsiWe same set of admihistrative problems under the same 
a " yotidittons. Each new. principal was given" memo- “pads, lotter- 
heads, i a pencils, and paper: clips, He was" instructed .. a oh 
aa morely t6 be the princtpal. He was not, to say what he would © es 
Mlo-4-he was: to dé it. He actually -weote memos, called meet- a 
ings, prepared* agendas, made notes in preparation for inter~« 
views, and the like. 
“The primary method ,of presenting problems in our * atudy 
\€@: was the in -basket test. Such a‘test consists of’ facsimiles of | 
f _ the letters; memoranda, ‘andvolher contents of the ‘in- -hasket ws 
such ag is found on every administrator's desk. Exhibits.1,.2,5 7”. - | 
~~ ; cand 3 in. your handout show some Injbasket terns and the © 
- responyes to them ntade by one of our subjects, Four in-basket 
" tests were presented, cach requiring a half- -dpy session. About 
*yhirty- -two problems. ay inctuded | in each: ipi- baskot; they were‘ 7 
Ghosen in the ‘light @f ‘il theordtical' formulation of “the Johof 26. 
an elémentary school ‘principal. - =. 
7s @ In addition to the desk Wark involved in the four In. baskets, 
esy thé subjects were required td participate dn committee work, nr 
soon’ observe the work of teachers shown if kinescopes, and, ** ~ 
react: to conference | situations. prebenied by means of tape | ae 
recordings. ee . 
“This is a very, brief ‘doycrt him of a “rather oluborate sitian ~ 
. onal test. The. participants“ ‘tell us that thé situation was *-., 
pie reallse and that they built up’, vivid imagey of the poople “ 
involved. a purticlpant received the same opportunities to 


cy 


» 


- © leari the background, arid cach was pfesented with the-samie” 2° \ 
ie » problems ander siden{ical conditions. Thorofore we can attrib: a eae 
woe ute ‘differences in ‘behavidr’ to the, participants rather than toh pees 
st vurlatlons anong jobs. as re re 


Bach principal left an envelope full of mefhos, letters, re- ae : 
', , minders; instructionsto hisYocretary; appointment calendars, ” < 


and the like, How gan this material be geored? wo ais 
The first qtep towffds developing ¢. scoring procadure was Bee 
2 to examipee, in-basket responges.to seo how segporidents‘differ, “er + 


7 As. wegult of such aan by a-number of ‘observers; *" 4... 
. { lat gag) pack ¢ toi curds was collected, éach card: containing ast 


Artem of sdmic kind of. ‘difference agoeved In the) way. the», 
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woe ie F ' 
swe yqu be willihg to call off {oe or returh thie note to me by 
huveday, indicating whether Monday, Sept ber 22, m Fae p.m., 
would be a sattofaakory time Sor a nect ng f ell th teavhere new 
to the eyeotan this year? Dr. tesa 4 and I thought. 1¢ might be « 
., gdodt idea to have en evaluation of ov orfentation program end also 
7 eo whether there era any questions thich they*have about the sehool , 
' eystem, after working in it for @ ahort length of time, Prinolpale . 
wuld be invited to thie arening too, te Mey a tatieg to atten me, 
swe feel the Principg) is « key le Orientation progam. «°° 


The Important thing now fe to find O fate thet-will ba aatisfaotory, 
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principals. pohavedl Tho chedy contained prises lke ' “com: - 
> pulsiye;” “postpones decisions,” and “mafes unwarranted 
3 assumptions.” A second source of ideas for dcoring categories 
oe from theories of" administration and leadership, espe-. 
ally Le ‘theories of the Director’ an Associate Direotor of 
ihe p 
svoritually all these modes of bchiavior were, collapsed, to. 
nrake the 68 scoring categorich shown in'Table 1 of yourhand- | . 
out, A scoring manual was written’ which gives appropriate 
definitions and rules for scoring, Each in-basket problem can 
be thought: of as ar opportunity to display the behavior des 
"+ serlbed by the category, On the score'sheet a one or a zero was 
cc > fecorded under each category heading, foreach problem, to - 
‘¢ Andicate whether or not the behavior did occur. A total of 132, 
7 problems was scored for cach ofthe 68 categorics, — + 
. The in-baskets were scored by cightscorers, each one scoring, 
; different half in-basket, Reltability was determined by cor- 
lating the scores obtained frem the odd-item scorers ‘with the, 
A pre for the even-itemn scorers, and correcting for length..Thus. , 
: é 1 rellabilities: reflect: both’ scoring decuracy und amount of 
~ % consistency in the principals’ behaviok, ‘The, reltabilities which 


‘ 


resulted vagicd from zcro to. .97, as Shown In Table 1, ES = 
/ 'Thgg40 categories which ‘are double-starred In Table 1 were 
= schosch fok use in the next phage of. the analysis. These cate- a 


ae gories had’ reliabilities of .52 or higher and a median reliabjlity . 
_of .78, ae Ue ores lgtlone of thesg 40 scores were computed, 3 
4 g and the,matrik of m rcorrelations was factored. Eight factors 
. Were retajnied; thoy ¢ “ej (8: almos¢ all of the common vari- 
ance among the, score’, Rotations were made graphically to .’ 
produce an oblique factor matrix.with simple structure, Table 
3 of the handout presents the category scores having. loadings | 
f .25 or more on cach of tho factors. | ‘ 
Factor A is called F. KC hanging Information, This inter rprata- | 
‘tion seoms quite clear ory the basis of the four scores with the 
highest loadings.’ We 
‘Factor B we call Discussing Before Acting. Aghin the inter- | 
. pretation is clear from the scores with the jighest, oe 
The “new structure” in initiates a new structure 16 likely to 
a committey or, other discussion group? the “proceduke” in’ 
arrives ata procedure for deciding ts uhaly to be a: discussion. 
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pe J. 
Factor C fag appear to suivole precipitate and gggressive 


action from the categgties concluding decision and terminal 
action. But the two following leads’ categories algo have high 
loadings. The acti6ns and decisions are thus likely to be made 


in compliance with suggestions; aa the name Complying ‘ 


with Suggestions Made by Others, 

Factor D has vory high loadings on swo catogorigs. The 
factor is called Analyzing the Situation because it appears to 
involve broad situational analysis of in-basket problems. 


' . 4 Factor’ EK, has loadings on categories which imply concern 


about: superiors and outsiders, and the negative loading on 


{delays implies prompt attion, We ogll this factor Maintaining 


Oryanizational Relationships. 

- Factor 4s Organizing Work, and it is shisbacuieleed particu- , 
arly by care in specifying in advance quite oxacWy when one's 

ork Is to be done, , 

Factor G is called Responding to Outgiders because the four 

Categorics with the highest loadings all have to do with people 
outside the organization. We might think of Phis:factor. as re- 
tlecting concern about community relations. 
‘ice His interpreted as Directing the Work of thers. 
Giving directions to sybordinates is likely to be done in writing - 
rather than orally, and courttsy is often uscd, apparently to 
#0 iy the blow. 

A secgnd-order factor analysis revealed two factors which 


. wo call X and Y. ‘The saturations gf the individual scofing cate- 


"" gorles Were determined,..and Yiose saturations of category. 


‘or 


scores Which are .50 or higher gre shown in Table 3. 
Factor X has high saturations on a number of scores haviig:< 


. to do With produotivity—number of words written, number of 


courses of action taken, numbeg of people involved, and Ad gn. 


mtpones and plans only, We call‘ this s@cond-or factor. 


= Amount of Work. Done|in Handling ‘Items. 


Factor. Y is a ‘bipolar factor with negative loadings on - 
concluding Prato and terminal action and positive load- 
ings on a variety of scores having to do with deciding how 
to proceed, getting informed, and ape, discussions. Hence 
the name, Preparation for Deetsion vs. ‘Taking Final Action. 

‘The actor ¥ Walling shown in Table 3 represent the OX. 
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= Fe s Normen Frederiksen ' 
tremes of a continuum which ‘has a rather ‘marked similarity’ , 2 
to certain theories of decision-making. Tabl@4 identifies six: 
theoretical stages in the process of decision-making. In the’ 
parallel columns are shown some-selected category scores. and 
their loadings on Factor, Y. The agreement bétween the theo- 
retical formulation and the erhpirical findings is’ ratheg strik- 
ing. This is, of course, no verification of the theorytf#at one 
goes through these stages {n- reaching a decision. There is 
clear indication; howeber, that a principal's ‘charkoteristic 
behavior ih response to a standard sect ‘of administrative 
problems can be described ¢n terms of his position on the 


asa of decision development. 


A ‘large. number, of other ‘variables—ratings, inventory, 
scores, ability measures, and go.o0n—was also available. The 
relation of the fagtor scores to these other variables is a matter 


- of considerable interest. Therefore the intercorrelations of 120 


f 


variables — computed. The variables iricluded in-basket 
category sco ial ability measures, personality inventory scores, 
ratings, intertyt measures, tests of professfonal knowledge, 
and biographical items. 

It is, of course, possible merely to compute the correlations 
between ‘factor scores and other variables; but such an ap- 
proach is versity misleading because of the fact that the 
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factors arg substantially correlated and hence contain variance _ 
'- which is shared by other factors. A factor-analytic approach - 
to the problem was adopted, at the suggestion of Dr. Ledyard 
- Tucker, which makes it possible, in effect, to‘determine the 
.  Felatlonship of each variable.to the -part of cach factor which 
. 48 not shared by other factors. For example, Amount-of Work . 4 
; , influences in varying degrees’ the scores fr all the cight. fac- | 
"4" tors; it would be desirable to learn the lationship between 
-, Ls varlables and factors with the effect of Amount of Work ruled _ 
‘, Out» The method employed resulted in the computation of 
coefficients which are pfgportional to the correlations with 


, the unique part of cath of the eight factors. ; ‘se 
4 The neuro follows: ithe 120 x 120 matrix was, , 


factored, An orthog factor matrix composed of the first ° | 
ten factors wds rotated to form an oblique matrix having a: v 
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- factor structure | as neatly as possible like that found for eae 
original in-basket factor analysis. Coefficients. were computed ,~ 
which reflect the relative relationship of each of the 126. 

. variables to each of eight oblique referenee, vectors, each vector 

_ corresponding to‘one of the cight in-basket factors. These 
coefficients are proportional to the correlations with the unique-. 
portion of the Cowesponding factor. Similar estimates Of cor- 
relations with s¢ecorid-order factors were compuited. We will 
have time merely to ‘indicate briefly a few salient ‘findings. 
ss Table 5 shows the relationships of several «cognitive 

measures with in-basket factors. We see that both off the 
second-order fa (Columns 9 and 10) have, fairly high . 
relationships wi the tests. Afnount of Work is related toa . 
variety of cognitive abilities, while Preparation for Decision 4, : 
relates especiallygo the School Administration test. The signs: * 

_ of the correlations in-Coluinn 10 show that high ability people 
tend to prepare for decision rather than to take a€tion. ne 

The other columns show relationships with the unique ~~ 
parts of the eight in-basket factors—the part Zemaining after 
the variance due to second-order factors is removed. The co-_ 
cflicients in the first eight columns therefore ‘cannot be attrib- 
uted to Amount Bi Work or to Preparation for Decision. a 

There arc a number of high positive coefficients, particularly —, 
in Columns 1 and 3. People who characteristically exchange 
information (Column .1') are high‘on Verbal and Nim x 

factors and on the test of school administration, The r a 
tionships with Complying «with Suggestions | «Column 
invoWe a completely different set of tests, particularly test 
of reasoning. Perhaps the compliance factor‘ is not weak sub- 
mission, but, involves logical evaluation of suggestions. ; 

Coefficients in Columns 7. and 8 are mostly negative. Those, 

"principals whd-Avere responsive to * pressures ahd who. 
were characterized by’ actively dire€tIng the work of thelr 
subordinates tended to be the-tess able principals. = 

Table 6 gives relationships between in-basket factors and =, * 

. > some selected ecores from Cattell’s 16 Personality’Factor In-., 

+. ventory. ‘The personality scores do not predict the sccqgnd-order - 
factors; but therg. are a number of high relationstips with . . 

« the unique parts of the primary factors. Look at Column 6, 

or sae Aes a =e lea try. to ‘maintain ae 
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7 sgatlzanlona relationships’ iend to be friendly, ‘adverityrous . 
rather.than’ shy, and free from” anxiety and nervous ‘tension. 


‘Look at Column.6.. Principals who, typically plan their work 
.- for apectfic Bays-and hours tend to lack frustration tolefance 
and to. be shy, syspfcious, anxious, and ‘nervous. Anxious: 


people appear to exhibit a compulsive patienn of behavior in 


handling in-basket problems. 


‘Table 7 shows relationships of in-basket factors with some ~ 
biographical information items. Again the second-order fac; © 


tors are unrelated to the predictors, but the unique parts of — 


* the primary factors -have: high relationships with certain of 


_method, of, factoring, which was employed appears ‘to be ‘a af 


“and ne their. relationshife with cnet measures. wg 


the items. Years of college training has nothing to do with-. 
any of the factors; but <dge, experience, and sex do. Complying 
with Sug ggestions (Column -3) is typical of inexperienced . 
young men.; Responding to ‘Outsiders nn 7) is charac: ‘ 
teristic of experienced old women. 


- The simulation of a standard-job in educational administra- 


tion through the use of in-baskets hag proved to be successful 


» as a method of collecting record’ of - administrative perfor: a 
- mance which’ can’ be scOred reliably; and which yields scores '.. 


which are usefultin providing a better. understanding of some. | 
of the dinfensions of performance, in such a situation, The * 
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ibaa M. W. se ot . . - 7 Models..of - : an 
A Fe 'RAVERS, Depart: i 3 on “ eo 
es ment of Educational eae v fos Teacher Rehavior Pet 
Paychology, Univer. J) | E in the Classré0om. ” 4 
+ sitv@g Utah fe wae a 
eS Sag, eek “oe ow: a a ee. as Pak 
bo sci 7 AR BUILDING ‘of models,.supposedly descriptive of what hap,’ ” ‘ 
“pens in American classrooms, ‘has. become an’ activity which) ~- | : 
_ Gecupies the’time of numerous persons from many different 7 
Sections of our culture. Most educational reformers begin:with © °° 
| _ the, premise that certain’ activities, of which they disapprove, 6 "J 
) oes +." pecupy much of, the’time of pupils.in the classroom, and that = °°: 
77 s+ ohanges should ‘be made whieh would. eliminate ghest actvie 
' ties. Those who: believe ‘that the schools are d@minated by 
ab _What they calla life'adjustment curriculum fall in ‘o.thisicdtes 
Unis, gory. af imaginative model builders. In ‘addition, there ate, 
>.) a +: theorists among:the behavioral scientists who also have shodels.. | 
(7 " ~* based -on certain beliefs concerning. what goés dn’ in ‘class 
| * ‘sy Poumise-They also advocate certain reforms designed to change 
wo! whhae’ they imagine to ocqur in classrooms. Most of these model oe . 
| *|/ + -bullders base their descriptions of: what supposedly, octura ‘in i.“ 
“‘@ ; classrooms’ on hearsay, an examination of textbooks ‘read py. /:" 
-,** students of education, and uneedotal material; brought hdine oa 
by ‘their children. A few of the more ‘eager onés make a few? i. 
visits -to. schools. -Erithusiasmy for: describing and’ attdcking © | 
." , “what. go¢s.on in hypothetical schoo|s:has ‘hard y been matchi 
vg" by enthusiasm’ for, collecting -datd “concerning ‘what actu@lly’ : 
Fg es on Taupene, Surely educational. reform ‘should: begin, not’ with. « 
2. at, Tmodels, of classroom: happenings ‘based, on. fmagtnation, ‘but.’ 
“se. on earefully and perhaps tediously collected data concerning ° 
ar a what actually happehs ‘in’ Am@rican: classrooms.. My. purpose : 
wo, 7 os here Ag to: present a,small amount Of. data concerning this; |: 
"3" 4 matter. based on what. must be judged tobe a:limited: amount. 
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‘| have been conducdig a as a 1 part of the Cooperative Research rin 
|)... Program of the Unitéd States Office of Education. Our project.) + © 
.--.,.* is coneémnedfwith the determinants of teacher behavior and~ = - © 5 " 
The data’ collection proggss has~taken us into elementary ~~) 
whe school where we have spent many hundred hours durtng the 
‘ lust year ‘recording the events that take place. Our project is . 
‘+ * woe concerned in-any way with the evaluation of teachers but* of 8 
we is designe to study teaching as a natural -phenomenon so “a 
. that some ‘of the variables relgted to Particular patterns of4 ~- 
re ,tqacher behavior may be identified. ~ oS Ps 
AN inevitable result of hig uy is that a considergbles 
= ee ae of data has been col sted concerning the practices 
f se rocedures of teachers in-the schools in--which, the sa 
arch has been conducted. While the primary interest in suc 
- -data isa ‘sclantific one, the materialts-also of general interest. 
: . “at this time “since: the’ public has become deeply concerned - 
. with what. is: happening in schools, The data presented here 5 
-« Tepregent: a Somnall . contributign ‘af knowledge concerning aes 
‘actual events lin schools which may be pf intdrest to reformers 
ie ayd.a starting point from which plang for educational change . 
;. might ‘begin. While our data are limited, they are still exten- my 
. sive compayed with most other: dita, Roncerning what: actually see 
ha pens in Schools. a. 
nat data wore: collggtedgin five schools within the Salt Lake OT eae 
Gh 8 District. ‘Phese: schools aré located in a*variety. of © 9 : 
*“differeheockil-cconomic nejghbortoods ‘as well as in different’: le 
.” spatts.of the town. The schools were not originally selected ; 
‘eto be teprdsentative of the school system although they are = * 
to a,cogsiderable degree, The only known diffeténces betwebn - + 
these: schools and the otHer schools of the, systerh rest inthe 
*" fact: thatty"n represent sgme of the larger units, In addition, =)" 
‘¢7-for tht purposes; of our research, we wished to include some :,’ 
a vf ..” teadhérs. who. word'knowr’to represent fre progredsive tradi. 
6s. ‘tion and for this reason we dncluded the*five lower’ gradef 7%. , 
| 7 de phe campus ‘school ut the. University of Utah. The inclusion i: i. ss 
ce oof the fatter grades probably loads our sample rather heavily ". ne ee 
. With teachers who: state ‘he they er a progréssive ¥ oo 
~ tradition, ‘This ‘is an important point to keep in minds | se 
The data to be all eae here consist of samples .of states ©... 
SP vionts made by ceteeten during, two, separate Visits con- x 
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‘ducted by qgch of ‘tworqbservers, Dr, Wallen and myself, Our: 


procedure was essentially that developed by Withall (1,) which 


ing each, minut interval until twenty-five statements had 
been collected.’The only types of verbal behavior excluded from 


_ our observation. were certain repetitive verbal behaviors as 


would occur;-for example, wien the teacher was reading a 
spelling test. Although an atteript was made to collect 95. 


statements during each visit'to“a teacher, this was-notfilways | 


feasible: Some teachers showed relativelf little verbal behavior , - 
and several hours -of observation were necessary Jn order to 
obtain two sanipley of twenty-five statements. Since the $wo 


"observers euch collected two samples from cach one of the 
- teachers, a total of 100 samples of nih ach were col 


_& broad orientation concerning the purposes of the research - 


lected on each teacher. Altogether, on the. 


teachers, a total 
of 8.300 sumples of verbal behavior was c 


llected. ° 


Rrecautions were taken to ensyre that the introduction of - 


arf pbserver into the classroom would -have as little influence 
as possible on.the teaching process, ‘The teachers were given 


tence, They were also assured that all records would be cog-. 
sidered confidential and that the results of. the study wou 

be reported back:to the partidipating teachers, They were also — 
asked to conduct their classés as planned when ‘the visitor 
arrived and not to put on any special program or démonstra- ' 
tion at the time of the visit. Preliminary visits were made to 4 


. 


and were assured that they were not being rated for cop 
d 


‘ each classroom until the teacher seemed ‘at ease with vgn’ 


observer, An effort was made to‘ know the teachers.on a ggt- 


_ sonal basis by chatting with them during the recess and before . 
Glase in the morning. A yreat amount-of time and effort was 


they could be at ease during thie observation sessions. The : 


paged on 3 ee 


devoted to developing good rblations witi’ teachers. so that 


project staff regurded this-as an excellent investment 5f time: 
‘Our géneral observation supports the polnt of view that 


: " required the obseryer to record the first statement made dur-, |)... 


_ these samples of verbal behavior are to a high degree repre. - 
sentative of (hybehaviors manifested by the teachers. Some " 


aspects: of behavior are inevitably missed by the technique. ‘A, a ; 
‘Blangé of approval*or disapprovalfwould not be recorded nor.’ ‘* 
. : ‘3 ¢ ae cos coe : . ‘ s ‘ 
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. would a elie a upil remark sind’ In a low whisper. Our, 
hee G impression is that most of. the" behavior of most ‘teachers is 
public verbal behavior... “+ 

.  Qne doesnot have to make an, & 

“Ht of verbal behavior to obtain’ gh over-all impression ‘of 
Ripe in these classrooms: For'this purpose a sample 
‘wilf'suffice. Hence we have’ selected from cach group of 28 
verbal behaviors a sample of one which in each: case was 

"selected through the us¢ of a table of random numbers. This 
provides a group of 332 verbal statements distributed over 

_g 83 teachers. A @higgification of .these mateanerts 4, aon in 


alysivrot all the® 8,300 . 


- Table... bo, 
ae 7 = aes - Tablet 
_Dietetbution of Sample of 332 Verbal Sehaviore of Tenehere 
.¢?.  Optogory of Behavior ‘Frequency Percent of Total 
» 
Thtorminu (ncademis) a] . * * hgstase 8% 
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There are a number of matters of interest to point out in 
f this table. First, consider the total frequency of occurrence of 
verbal behaviors in each ane of the categories. The most fre- 
quently occurring is that of telling the pupil what to ‘do, a 
traditional practice which has had many advocates through ° 
. the ages. OF course, many contemporaries who'think about. - 
Seen eerareinath believe that more of pupil -behavior | * 
shduld be self. initiated, but the fact tp thatgin ‘our sample +} 
the teachers devoted ‘much of their energy to directing the |": 
+ academic work of the-puplls. A second: point to note.is that’ 
the’ next -most frequently occurring form of teacher behavior 
ly that’ inyolved in a questioning process, This #oflects the 
extensive use of the traditional recitation procedure. The third . 
most’ frequently occurring form of tcachpr behavior, outside 
. Of performing classroom management functions, is that of 
» providing information. ‘This also is 4 traditional function of, 
‘teachers, although orfe about’ which many-questions have been “8 
raised in recent educational literature. In contrast, the cate.) 
gorles. of teacher behavior which the progressive education - 
- . movement of the thirties considered to be important: show 
. | dow or zero frequencies of ‘eccurrence. For example, it is a 
7 . ware event fOr a teacher to delegate to a pupil'a decision- — 
4 making function. Only 1.5 percent.of the teachef behaviors 
‘fall intg, this category, The emphasis on personal and social ; 
adjustment, which some claim to a focal weakness in our | -., 


> 


_ education system, is cympletely lagking in our data. Indeed, 
the impression giver? by the. table i that of a subject matters 
oriented curriculum taught by méthods, which have been 
go teadittonally practiced and which the public widely accepts. 
_ ‘The data agree well with our goneral observations. We noted 
a high degree of emphasis on the learning of subject matter. 
Paps 4 . Tho capacity of teachers to handle a rangé Of subject matter 
' which covered such diverse. topjcs as meteorology, physics® | 
physiology, geology,sas well %s the more conventionally taught 
areag of knowledge was impressive. ‘The excellent. prepdtation 
© 4... Of the teachers int this respect is damonstrated by ble 
ese that. out‘of 8,400 revordéé dtatements of teachers, theres is = 
_,. ‘only gne subject matter error. ‘This is en enviable record ;and’ ~~) _ 
‘¢ . 4 , One which presents a standard which T would like to live up’. - 


*: “to in my own teaching though [am suro-I do mot. The trade 
€ a ‘ page 4a y aoe ; ae t : tg } : . s : ; 
r) | ¥ - . . 


 tlonal subject matter of education Is amply represented in ouk 
sample, If two expresaions. giving an over-all picture of the 
. Classrooms of these teachers were to be chosen, I think the 
-” two most appropriate ‘would be that they were subject matter- 
_ oriented and businesslike. — 
Psychologists, who have recently joined the ranks of educa-— 
tional reformers, are likely to'look at the data much as they 
have sometimes looked. at classrooms and jump td the con- 
clusion that teaching does nat provide adequate reinforcements 
for effieient learning. | do not think that such a conclusion 
is justified, The reinforcements operating in the classroom 
*, are very difficult to identify. Indeed, unul I had spent a few 
hundred hours in classrooms, | did not become aware of the 
wealth of reinforcements that are operating. For example, in ° 
most classrooms if a teacher does not comm®nt on a student's 
answer, the pupil will assume that his answer was right.‘ 
Silence ts probably the most commonly occurring reinforcer 
+ . and hence docs not+appear in our ‘data. Othe? reinforcers, 
| which are not castly observed, are those built into the books 
and workbooks, A child stumbling through an clementarf- 
reader ts reinforced for those. efforts which result in the emer: 
yence of a meaningful story. Again, many cues may be 
= present which indicate to a child Solving a mathematical prob-. 
¥ .. lem that he fs right» Psychologists who visit classrooms and 
report an impoverishment..of reinforcing agents: haye” been 
looking in the wrong direction. | syspect. that psychologists _ 
are prone to look for the kinds of reinforcing agencies that - 
~ they commonly. introduced into the laboratory and since they 
a cannot see these itr the classroom, may assume that few rein- 
»  forcements are provided. Having arrived: at thig ‘conclusion 
- the next step is likely to be that of designing a classroom 
', «Which reéembles theif on laboratory and in whieh they feel 
‘ ‘cofnfortable. This kind of educational reform is andi gous to 
.- . '« the efforts of ar~imaginary psychologist who, after apending 
p 4 lifetime studying the behavior of chickens, spent ‘his years — 
of retirement wating a book ‘ony: w to teach rats to peck. 
“While the data refer to a single Spool system, my impres- 


sion is that these classroome are very similar to those which: «.. 


~ Thave visited elsewhere, This suggests to mé that teaching Je 


- . porformed largely {In terms of. cultural tradition, Teachers -- 
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= ‘ have to satisfy the expectations of parents, ‘children, school 7 
boards, and other conununity ‘groups, if they are to retain 
" their jobs, and these expectations may well play a much more | 
significant role as determinants of teachér: behavior: thando 
all of the verbal transactions involved’ in teacher education, 
Our data are not consistent with the notion found in populay - 
magazines that the tdeas of John Dewey have produced a 
radical change in teaching practices. As a matter of fact, the 
influence of John Dewey is difficult to’ discern in our data. 
There is no evidence in our data to su port the notion that 
the teachers’ In, odr sample were particularly concerned with 
a life adjustment curriculum of the type that has been criti 
-cized in the press. The behavior of our teachers shows a central 
concern for intellectual development, and one has to look hard 
mong our recorded 8,300 verbal statements to find some 
which have-to do with the emotional and social development 
of pupils, Certainly less than oneyhalf percent ‘of tcachor 
behavior, recorded in ur study has anyffing to do with this 
objective, The rare teacher behaviors that have to do with 
this objective were gencrally initiated by some crisis: such as 
occurred in ong classroam where the tea¢her noted that a. 
foreign child who did net speak English was being mistreated 
5 by ‘his classmates. The educatidnal process, as we haye ob- ° 
Served it/in our sample of tvachers was almost exclusively 
intellectwalistt: in both purpose and execution, and this despite 
the fact that we went to considerable pains to locate and 
« “include some teachers who supposedly taught:in a so-called 
progressive tradition, "| | @ a 
Our data leave us with thg impression that the behavior’ 
patterns of the teachers In $@r sample’ represent a teaching. 
tradition which is probably highly stablo and difficult to ©. :. 
change. The influence of teacher education is seen: largely: * 
4 in command of subject matter, an area in which our teache 
Showed: oxtraordinary competence. Those aspects of teacher , 
+ * edugation which have to do with teaching methods show much: _)- 
“¢ _ less discernible éffects, which perhaps js hardly:surpriaing ia - 
‘vy Wlew.of the fact that. such. courses wengrally represent only -— 
‘ . . -  ahout 25 percent of the teacher-training curriculum. Also, pro- ae 
_ . *. fosstonal courses in education are faced with the very difficult | a 
oe task of changing already: ‘Woll-estdblished concepts of edudi® 7 ade 
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tion and habits of dealing with the young. I am not saying 
4 ‘that professional courses in education are not important, for 
*" . ‘+ | am convinced that they are, but they attempt to undertake. 
a task which is fantastically difficult to‘accomplish. © ae 
_ The data which we have’ presented ‘and the total Body of » 
data from which it is rena represent only a small portion. ° 
of the data needed. for the building of what one might hope 
‘to be a useful model of the behaviorfof teachers as it qaccurs ~ 
in schools, ‘There Is a great lack of data concerning thé incl- 
dence and operation of reinforcements in the classroom and. 
the extent "to which reinforcing functions. aré and cgh be. 
_ defegated to workbooks and other forms of equipment. Data 
on this’ problem may well‘ lave to be collected in miniature 
experimental situations rather than in the classroom itself. 
We ‘look forward to having available an increasing body of | 
~ data’ which can be used to develop a theoretical medel’of how 
teachers operate. Such knowledge, as it evolves, will slowly 
equip the scientist with some of the knowledge which h@ must! ' 
have if he isto introduce successful changes into the gem. 
° REFERENCE | . 
1. Withall} John. ‘The’development of & technique for the measurement 
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_ | Note: The research reported herein was ‘supported ‘through “the Co 


Department of Edu- 


cational Psychology, \@ehavior in the 


dge-mates and older people. Thus, talented ‘behavior ifivolyes 
both personalM-significant and socially-valued competenoles, 


including signs'of creativity, recognized as such through per-: - 
_ formlances or products which can; be assessed by oghe perdony.., 


The basic, assumption underlying the ‘Human Thlent ‘Re- 


ordinary is done in guiding learning experiences; process 


“we, term “telesis,” the behavior |of most individuals attains a 


.Canson MoGua, | The Prediotion of Talented: 


The University of Ne Junior High Scttool \ 


‘ Beatch Project is that! if factors aych-as innate ability be held - 

~ constant, talented behavior {s ma red and becomés organized; - 
or structured, and o some extent predictable, as a conse-_ 

‘quence of. the educative ing Unless somethirig out of the 


kind of crude stability or invariance which ‘makes’ if fire. © 
dictable.4 This paper. presents an approach to studying and. — 


testing Meas about talented behavior in school and, eventu- 
“ally, if experimental ‘seltings, The major outcome of the. 
. research has been the development of “factor variables”: which: . - 


Pee 
" 
4 


operative Rese Program of the United States, Office of Eidu- 


VW 


: > a . . 
. * 4 3 r ‘ 
: , ’ 1 é & a jee po 1s, ws eo p 
5 ad cn) ie By mea est sepia Je at So ‘ ® 
ae * es ' . : my, é ar a ” wae eg Om et ' Voie 
oy a * ’ oot “ r : 7 an. ‘7 a ~ ‘ 2. > 
a * ’ i ay ’ Ai * ‘ bs at 


‘cation, Department of Health, Education and Welfage: In this 
 firet etage-of the Human Talent Research Project, factilty asso- 
- + ¢lates were Ralph BY Duke, Benjamin Fruchter, Besthan ‘N, Aa 
Phillips, John Pierc@Jones, and. Jackson B. Reid," tvéearch . 

a, <eeeodlates on ‘the project. staff ‘were Edwin Hindsman, Earl - 
toy foe cdennings, FJ. King, and George MoBeeicor 6 
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_A ‘TALENT 18 a pre-eminent. aptitude, or a superior’ ability, 6 “ys 
- either‘natural or acquired, or a capacity for achievement or \ : 
success. Ih school settings, “human taleng, refers not only.to . \ . 
the various kinds of scholastic ‘aptitude an@™Meademic achieve. . —\ 
ment measured by objective instfuments Wd evaluated by \ 
teachers, but also to other forms of behavior prajped by one’s |.’ 


é : : ; . . - - —_ ae r _ 


t combine Indicators of talented behavior. Two sets of factor _ 
. varlables, one for the general populatian with modifiers for © 
. the two sexes and the other found among highly-intelligent 
students, have turned out to be: surprisingly stable'and valid 
predictors wh a good deal of SAplanAlony value. ‘ . 


Nature of Tisniea nvHee * 


A general: proposition ‘abgut the nature of talented behavior i 
- provides a research el set forth schematically in figure 1.“ 
Talented behavior maybe regarded as a function of:. no 
. (a) pertinent cognitive, perceptual, and psychomotor abili- 
1 “ties (Bloom, 1966; Ferguson, aie Sulton, 1956, 
. 1959; Wilson, 1958); 
(b) elements of personality: and: inbilvation, eapecially _f 
. ° expectations. about one’s behavior and the ‘probable . 
‘responses of other persons: to it (Atkinson, 1057; Sears, 

1961; Stern, Stein, & Blofm, 1956);- ee] 
~(c) ways in which an individual is valued and has pres- — 
sures infposed upon him or her by others, such as — 

parents, age-mates, or teachers, in various ‘settings a 

(Elkins, 1958; Haggard, 1957; McGuir® 1958). aa 

a These three classes of variables may combine in different . 
ways as a consequence of (d) sex-role identification and the — 
sex-typing of socialization pressures, and (e) the context of 
behavior, especiglly the varying patterns of educational expert- 
ences and role expectatiogg in. different school locations .. 
(Gocthals, 1958; Ferguson, BGA sere 1955)** 


Gathering and Processing the Data 


Frankly speaking, tho research tearn faced a monumental task 
. of guthoring, organizing, processing, and muking pense of a 
_. «wealth of data in three years, from the early fall of 1957 to 
tthe late spring of 1960.. ‘Decisions. as to what instruments 
; ‘» were to be administered Were made by a committee ih the +! | 
, ‘spring and summer of 1957. A number of test# Were revised — oy 
. ° for use with adolescents and other deviges were*prepared* = 
' (MgGuire & Associates, 1960). The admirptcaion of ae . 
- ments was carried out largely by members’of the project st 
‘and faculty associates with some valuable, i ila from 7 
> the people in ue four ‘school systems, © 
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By means: of several kinds of multivariate analysis in pilot — 
studies,,variables judped to be most pertinent to, the study’ 
of talented behavior were: identified, (Hindsman '& Duke, 

- 1960). The tedm used IBM cards and, later, tapes to organize 

. ‘and -storé data, Programs were written for basic processes — - 
such as transforming the original values punched on cards| 
after scoring into another deck of cards. with stanine values. 
The reduction of raw data involving 140 moasures into 
 stanine values facitaed variance, factor-analytic, and mul- 

Uple-regressidh ‘analyses using IBM 650 equipment.. _ 

a i oe 

4 a 3 ” Table 1 
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a a r) ; ae WE 
Regressian of @PA Teacher Evaiuations tor Grade Vii upon 
Selected Indipator Variables by Gommunity and Sex Rote 


(Decimal points omitted) 


‘ 7 . Communities 
._ <= e Indicator - A, B® +» ‘0 “ yp : 
; Variables” _ MF’ MF MF _ 
my ° ie : ° ‘ 
Sample populatisn, No NGL 187 96 a 988 941 aoris 
: / Regression variance, Rt Soya 59 ya 170 0488 65 
Multiply corrdration, R 4" 60 70" 80 ae 80 102.81 ae 
. OTM 1 Function, . "19 19 esa 
: ; DAMM Mental Function, 6 q -if 4 % 
e » Goathit Transform'n. ||! » 4 ‘ og 9 19 
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Then the most atinniiaung.a and : peadupaive: steps in the 
processing of data began. By means of a series ‘of factor- 
analytic studies of measures from 1,417 boys and girls in the 
seventh’ grade, we were able to.map out the different ways 
in which variables combined ‘oj sort out’ and portray achieve- 
monts, abilities, and attributes. We also sorted out ‘the mean-: 
ings of descriptions age-mates gave of one.another in response>~. 
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to 46 nomination itemb. The analyses indigated that five socio- 
. Metric factors were common to.both sexes and, ‘consequently, ° . 
wo could use a smaller number of factpr scores to represent 


appraisals by peers. Repeated meoasurgmer 
' afd nine for selected variables’ proVided distributions for 
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1960 In + Conference on Testing Problems | i 
to 4 studying reliabilities as well as criterion measures ‘for.the | 
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_‘*, prediction studies (McGuire & Associates, 1960). : . 
“Two complementary techniques, multiple regression, and 
* | factor analysis, were employed to identify the-gignificant indi- 
cators of talented behavior and-to reduce ‘them to ar even 
smaller number of meaningful factor variables. As. shown in 

' Tables 1 and 2, multiple'correlation goeffictents ranging from 


R = .75 to R = .86 were ‘obtained to represent. the ‘relation- 


i sh ps between criterioti measures, such as teacher evaluations 
_ and'standard tests» to various combinations of the 32 indicator 


variables. Table 2 Would lead one to infer that measures of - 


. " strength of association ‘wero mot quite-so high when data were 
oN combined across communities for boys and girls. Nevertheless,, 


_ 1 of the variation in the different kinds of valued performances. 


The results encourafed the research. taam to: take the next. | ; 
step of reducing the 32 promising indicators to a smaller num: _ 


ber of factor variables which could more effectively explain, 
¢ Yet'still: predict with:some precision and confidence, different 
kinds’ of ‘talented behavior. As. summarized in Table 3,, this 
was done first for the total population of 1,242 students who 


had completed the hecessary insttuments in grade sevén! In | 


‘ . = the analyses almost invariably. explained 50 per cent or more : 


_) 


7 Blan population, we sorted out five factors common to . 


oys and girls, two specific to each sex, and two characterizing 
devignt behavior. Table 3 shows the, factor loadings and the 

_, Optimal weights for the componeniindicators of talented. 
beliavior, the latter being determined: by aout regression 


thethod. Using ‘the optimal weights and a suitable program, 
ge to bé corisidered je-the prediction studies, . 
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and achievement of gifte ildren-by Haggard 1057). The 


_' first -step. was’ to construct 4 master intercpreelation matrix - 
'~ “ for 32+ promising: indivatorg §f -talented behavior and six 


"criterion measures, using valaes obtained in grade seven. As 
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shown in Table 4, the'regressions of-criterion, measures upon 
_patgntial indicators were computed using ‘An ‘iterative, tech- 


nique (Greenberger & Ward, 1956) which had a stop criterion 
* . to avoid overfitting the regression line, Because original stanine 
“, Yalugs were employed, mo$fpf the cognitive distributions were” 
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. ~ ment of factor vatiables. Correlations among the 32 indicators 


. +, provided a matrix for Hage raction of centroid factors which 
‘y . 2-were rotated to an orthoggnal normal varimax solution. 
oe Kaiser, 1958) by@@ppropriate ,programming. The varimax 
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* The explanatory | nd\ predictive values of the factor variables 
were explored bybdetermining thgir relative weights for and _ 
wtrength of association with two kinds of criterion measures, ears 
_, - academic achievements and agg-mateappraigals. “Results tiave “~~ | 
_* been summarized in Table 9 for the general - populagon : 
fo "CN =y740) and for. the -highly-intelligent HMF studehts 
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_* | « tended ne school from grade spven to grade fine. The “modal 
« —_.Welghts’\shown In the-two tables..were estimated from the 
y* * - original data (McGuire & Associates, 1960) andshow.thecon- == *" 
tribution Which a factor variable makes toward the explanation ' 
“ . ,afid/or prediction’ of sorne kind of valued performance over, 
~ 48 specified périod of time for: a ‘specified population. The 
+ de@ading magnitudes should not be venfused with correlation «| 
> coetHclonts. They are multipliers or beta weights of sets of 
_ factor scores for persons which yield the most efficient ex- 
. Planation and prediction of a specified, kind of behavior. In* 
cach instance of @ significant change th ‘loading magnitudes, —— 
from Zero to a substantial weight ora shift in signs, two . 
__» @ntries have’ been made in the tables. Thus ‘the weights for -, 
- . factor variables may be viewed as being relative to one knother _ 
“¢, for @ given criterion measure, and comparisons can be made 
_ across different types of evaluated bghavior. Mogeover, changes 
over’ time due. to age and learning experiences or: different 
_ +«. forme of a criterion measure readily can be identified. 
‘". + As shown ip ‘Tables 9 and 10, the weights attached to the » 
factor Variables for each of the criterion measures. were differ- 
ont for teacher valuations, the several kinds of objective = 
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‘ each set t a criteriop/the loading magnitudes of a majority 
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"The severith-grade factor variables *apparently turned out ‘to By 
be “a/muth more standard article” (Guilford, 1984, p. 524) 
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than different Neth of indicators with fluctuating. combinations 
«such as those ‘shown In Tables 1 and 4. d. 
Another shert cut completed our tests of the vali y of” 

a - factor variablgs since we did not have time for cross-validation | 


deviations, from regression, with appropriate degrecs of free- 
dom, when paigs.of multiple correlations for the same criterion 
were compared. In grade seven, the sets of factor scores were 


separate sets of: ‘AndidRtors. Instead ¢ of. shrihking, /howeyer,- 
the multiple correldttony for the predictions of performances 
.+ ‘bh grade nine cither remained similar to or increasetl ‘signif- 


STEP Science for the general population, Where forgcasting 
efficiency deereaged significantly. For this criterfon casure, 
the coefficient of stability of performance over two forms,” 
y d 2A, wis, relatively low (ty <2 .45). Part of the insta-” 
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population did not begin the formal study of a science 
ghcir, last year’ of" junior high school. -In general, the 
pt variables served to explain a substantial portion of 
AG variation in performances, on the STEP instruments and 
showed improved multiple correlations for the advanced CAT - 
tests of achievement. Krom the foregothy analyses, we con® 
cluded. “that the factor variables were reasonably valid and 
| ' stable measures which served to explain a large proportion 
» Of the variance: in several kinds of talented BOnANICE, both 

~ . academic and non-academic, 4 


The factdr vardables develo 
school population in Table more fully described in 
Appendix 1, fell into‘ three categoria Five “were common to’ 
both sexes and explained a good deal of the variance in dif: 
ferent criterion measures, “Tha were cognit ve approach, . 


Reaulte and chr sno 


peer aliyytlus value, and absence of negative valuations. Two 

were specific for cach sex and often served to: modify the 

+ — gualfly of performance on measures of talented, behavior; 
: namely, anxioug emotionality and antisogial, wariness. for boys, . 
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not as strongly yelated to -the criteria of talent as were. the | 


cantly over those, m grade seven."The” only exception .was 


the goheral junior high 


~ dlvergent thinking? socially-ortented achievement motivation, 


studics, Thistwas a test of shrinkage based upon obtained 1 


$ atteibuted to the fact ‘that many students in the *, 4 
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‘and welinitive dependency and ‘perceptual defenses. for girls, 
“The remaining -two ‘variables were not linked to either sex 
rgle- and invariably acted as guppressors' to. explain less 
‘effective behavior; namely,. practical-minded toug iness and 
anxious-de pendent resentment. 


As shown in Table 9, the ways in’ which the foregoing factor 


*. variables combine vary according to the criterion’ of talented . 


i a 


“behavior. Patterns for boys‘and girls differ in most instances. o - 
Two. tend to have high relative loadings, gognitive approdch — 
‘for standard tests of achievement and peer r Stimulus value for 
agermate appraisals. Both are strongly represented ‘in féacher ° 
eyaluations. The other variables serve to aeraee ‘modify, : 
or suppress the ones with major weights. , 

'The HMF factor variables, ‘found .in -the study of me 
intelligent, junior high school students and summarized 
Table.G, are not the same ag those for the general population 4 % 
“They also are more fully described in Appendix .1. Three are’... 
tiie. in nature; namely, convergent thinking, divergent: °° 
thinking, and symbol aptitude. Two represent positive and |: 
negative elements of the makeup of personality and expressed 
attitudes, ‘personal stability and antisoctal attitude. The re- * 
maining pair combine peer valuations and characteristics of 
"the individual, age-mate acceptance and conforming behavior, 

The seven HMF factor variables'serve to explain a signifi: 
cant and substantial amount of the variation in teacher evalu: ~ 
ations and ia-scores achieved on a number of standard tespr - 
by the potentially-gifted students. Cegnitive abilities, person- . 

- ality attributes, .and peer appraisals dg not have the same 
patterns of modal weights from one criterion tq another: in 
“Gable 10. Therefore, one would ‘infer, that pomewhat, different 
combinations of characteristigh dre necesgary ny uccess in 

_the various subject matter ardagy ove tially able 
‘ students. Moreover, the pattems of weights on the factor vari-' 
ables for the’ samme measure of talent@ behavior are not 
necessarily the same. for Rove | and girls: five there ‘are some 
similarities, but one has to accept the Conclusion that there 
aye some important dux-typed differences and there are some 
. shift over stime, These differences have: an influence dpon 

the manner in which persons of cach wax ee pia cope 
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with various kinds of valued. performartces in the junior high. | 
. school grades. : ¥ ar £9 


oe  * ae r - a «, ‘ 
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By and largd, we believe that the major outcome of this - 


research is the development of factor variables which fit a 
model. and serve to ‘explain. and probably to predict several 


., kinds of talented behavior in. the junior high school years. 


Each set ‘functions somewhat like a differendal prediction 
battery (Horst, 1954.). All clse being equal, talented behavior 
is brought about by a combination of innate an@acquired 


-~ ¥ 


abilities, augmented or suppressed by clements of personality © 
and attitudes toward the world of people .and. things, and.“ 


facilitated or modified by the expectations and the pregsures: 
imposed by peers: and older persons. But two other influences 
are at work. One is the expectations and pressures associated 
with sex roles and the other involves the contextwsetting 
in which learning takes: place and behavior’ decurs. These 
.clemonts ‘of the prediction model can be represented by factor 
vatlubleg’ which, so far, have turned out to be stable and valid’ 


“ , measurgs. 4 : 


a @ evaluated objectively. By and large, the criteria employed in * . 
"this, regearch place great value uon gputude ith symbols 
ity 


n ; is ss 4 A 
, Perhaps our greatest concern is with the clio measures. 
regard as the 


They do: not quite, represert what we woul 
most appropriate ‘criteria of talented behavior, although many 
are commonly used to evaluate different kinds of performances . 
in school settings. As Michacl suggests (1957), there should 


. be 7 careful exploration: of ¢riterion. development so that’ 
a 


original thinking and desirable kinds of. creativity can be 


and convergent thinking, ‘the abi give the one “right 
, ariswer.” With rare exceptions, the data‘ indicdte that’ boys 
especially are penalized for divergent’ thinking in the 4unior 
high school years. Girls secg to be rewarded more frequendy 
by their teachers wien’ they are more resourceful and .[ess. 
goal:bound. Another concern: has to do with the degree to” 
which? various forms of. conforming befavior appear to be 
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\- . Desortption‘of Factor Variables 


‘TWO Rts of factor variables were developed in the study rof talonyed ’ 
‘. behawor-in’ junior high sthoels, One set omerged ‘ffom the analyses of: 3 
data gathered on 1,249 stddents, 634 Boys and G08 girls, of four Texas | 


vam 


same high school populations. ae: 


Facron VARIABLES FOR Junion Hicw ScHoon:Srupents | 


A sect of elevon factor variables served to explain and preditt the vari 
ous kinds of -talagted behavior observed among dll junior higt #hool 
students. Fuctor-analytic techniques were uted to sort. out ‘and map 


. * the sevoral kinds of abilities and other attributes. Then, for each of thé 


factor variables, a multiple regression procedure waa ertiplyed to esti. 
‘mate the optimal weights of each. of the indigators of, talented behavior. 
A. Cognitive Approach. Efficiency 1’ attending to and concentrating 


upon verbal stimull during,the learning process (STEP Listening); 
recognition of appropriate answers to problems involving spatial rela- 
tions, verbal concepts, logical ¢md numarival reagoning (CTMM); ability - 


to shift the. function of a part of in object and use ¥t in a-new way 


Mnities beginning 19 the seventh grade and followed through the. “ 

_ filnth grade where only 730 were: still: at the same school location. ©.” 
- The other set wis dorived from data on 213. highly-in@Mligent (HMF). 

or potentially-gifted adolescents, 120 males and 93 females, from the | . 


(Guetalt Transformation); capacity to visualize and formulate complex 


“~ 


@ given time (Common Situations). 


. C, Soctallp-Oriented Achlevérment Motivation. Acceptance of echool 


and, cultural standards (J ); soholastic motivation, acapmpanied 


by offective study hupits, and attitudes (88HA); representing origself'aa 


a stable, confident person (JPQ 3); four additional indicators, °° ' 
D, Peer Stimulus Value, Acceptance by one's peers (party with, like 


to be like, work with on a school problen), expected. to: bean effec. | 


tivelyfunctioning person (enjoys everything, self ‘confident, shows 
Initiative); seldom regarded as a quiet, dependent individual, 


E, Absence of Negative Vuhiations, Not necessarily recognized or 
. aucepted by age-mates but seldom named as a neégatiye model (not 
_ .- like to. be Uke, not ask for ys hot party with); seldom nominated — 


by peers for déviant behavior (@islikes school, has to be told, gete by)., 
Factas-matching techniques showed thdt the fivy foregoing factor 


‘variables were common to both boyd and’ girls. "The next faut were. 
apecific to males and feriales and two were identified for each sex, 


7? 
‘ 


F. 


F J ‘ ? ; : > “ i ve Veg 
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conceptiond: from figural materials (DAT Mechanical Reasoning); five 
» Other indicators have mich emaljer weights, - m a 
_ B, Divergent Thinking. Abtlity"to think in different, lees goalbound . 
» dirdottons ‘demonstrated by going beyond what. ie given and extrapolat- 
: ing outcomes (Consequences), awareness that problems exist (Seeing 2 
‘ Problems), and calling up as many ideas or responeeé as ‘possible in’ 


eo 


| 
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F. Anxious: Emotionaltty (Boys). Self reports indicate “a motive to- 
avoid. failure,” especially in ego-invoWing, threatening, or stressful yt 3 
8,” 


, tlont (CMAS Anxiety); misperception, inability ta cope with preasu 


and lack of a sense of identity (CYS -Maladjustmont); emotional sensi- 
tivity (JPQ 1); reacts emationally (JPQ 3). ‘> 

‘G. Antisoctal Warinegs ( Boys). A avt'to mcopt authoritarian iltndes. 
and awareness of corftrots exerted by authority figures (CYS Author/- 
tartan); somewhat alienated from:others (CYS Negative Roclal Orionta- 
tion); negative toward teachers and critical of what is expected in 


- school (CYS Criticism of Education); perceives’ symbolic materials 
“quickly, even from partial clues (Sho ‘ords, Mutilated Words), 


H, Sensitive Dependency (Girls). Kxprosses* authoritarian attitudes 
and accepts control of authority figures (CYS Authorttarian); emo- 
tlenally sensitive, tender-minded (JPQ 1); complies with most school 
and cultural standards (JPQ 8); quiet; serious (JPQ 11). yO 
of. Perceptual Defenses (Girls). Originally named “Perceptual-Motor 
Skills’ but reinterpreted; identifies*objects with parts inissing (Gestalt 
Completion ); -careful, to represent self positively (CYS Neg. Social Ori- 


he remaining two factor variables were identified in the ie 
cators for the total population: Neithyr of chem was correlated with 


a “fest quick and careful reactions (Dgtting, DRT Reaction Time), 


in 


_ factors obtained for ‘boys or girls sepdrately, sg the attributes which | 
they combing could not ba linked tpither sox role. ‘They wore regarded . 
as. moderator varlablés which? ombitied cognitive. and .noncognitive . 


measures (J) or giements 1) rsonality and exprosned attitudes (KY. 
J. Practic al-Minded Topghness. Represents galf‘as tough-minded and 
scoffing at emotions £JPQ 1); probatyy has. mechanical “experience . 


- and responds to pracfical problems (DAT Mechanical); lacks the’ ‘ability 
"to -percolve and-Cope with symbolic materials (Mutilated: Words ); low 


' HMF Faction VARIABLED Uo cy wh ‘STupENTs 


fear of failure (CMASAnxidty); cooperative, res 
‘ communicate (JPQ 9), five additional indicatots with lessor weights, 


sped of Closure on word units (Short Words). / 


“KY Anxious*Dependent Réenentment. Belf ropurts indie ate underlying: ' 


anxiety (CMAS); exprénses -duthorttarian, beliefs (CYB); distrustful, 
egocentric; and -redentful (CYS Nog. Sucial Origntation ); desurgent, 
passively awaits what is going to happen (JQ = 


1, Personal Stability. stabithty or egofst wae. (JPQ 3); “low 


hed, and able to 


2. Divergent Thinging. Ability to think in different, less goalbound 
directions marked by ideational ‘flucncy (Commop Situations); con 
ceptual flexibility (Consequerices); sensitivity to, or awarepess that 


problems exist (Seeing Problems); ,some originality, or spontaneous 


flexibility (Unusual Uses). 
3. Convergent Thinking. Ability to channel thinking to one unique 


coriclusion by megns of eduction (DAT Mechanical Reasoning); com: 


4 


prehension (CTMM),’ redefinitign (Gestalt ‘Transformation); and 


¥ attention to ,sthmull jot a verbal nature (STEP penne). oY 
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4, Symbob Aptitude. Rapid recognition of symbolic upits (Short 
- Words); facility with verbal materials, word fluancy (Rhymes); “per... 
: ' ception of aymbols when given only ‘partial clues,, synbolic dlosure *. 
"  (Myptilated Words); three-additional indicators’ ‘with ‘lebder weights, 


» , tates; acceptance of authoritarian’ belies: atid’ ewareneas of controls” : 
4 by authority figures (C¥8 Authoritarian); ‘seldom yegarded as deviery 
: in bohavior by peers, Pe a | arr on 2 AT 
i, oe 6. A ale Acceptance::Acceptance:by one's peers as a persen ‘to. pare 
oe be liker be with; cxpected t6’be an offectivg, mentally-health) person; (4 <-." 


‘ able to cope with préssures and: ‘possesses a” | of identity (CY8 
P Personal Maladjuntment 5. so -_ y 

va ,* 37, Antisog (at! Attitude, Toughminded outlook (JPQ 1); critical of 

ey —achool and teachers (@QYS Criticiem of Education); seldom regarded as 

_ » 4 quidtly ddpengent individual; somewhat alienated from other people 

: - (CYS Neg. Sotial Orientation 0° 7 ? 
- : ot 
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LADIES AND GENTLEMEN, | feel very brash indeed in appearing © 


before you today. As: somv of you. know very well, .1 had the 


" privilege of standing on this very rostrum just the day before’ 


yesterday. I enjoyed the experience, but my resources are not ent ‘ 


‘such that I can successfully emulate the activities of our presi-. 


dential candidates and speak daily on a variety of subjects. . 


What I have to say: this afterrioon is.known to all of you: - 
__ There is nothing new about it. I propose tg factor,no matrices. 


I shall cémputé no coefficionts of stability of performance, but ~ -— 


I do applaud those-who can and do s h. things and obtain 


the fact upon: which to’ base their.conclusions, - for. surely 


We make progress onfy in the measure in which we determine _ . 


the facts solidly. ard | proceed’ to reason to sensible 
conclusions, *. - >= 

Unhappily, since education is a subjett which has pre- 
occupied the mind of man for centuries, ptobably ever since 
the cave man era, we have gmany theorics of éducation. Some 
ure Spun out of tho gossamer of imaytnation and prejudice. 
They .are always giver wide distribution in the public presg. 
These: theories are not based upon facts, they are based upon. 
supposition and prejudice and we who work In education 


‘have to keep our blood pressures down, continue to ascertafh 


facts and ‘to think with the 
order to deal’ adequately. 
What causes this cer ua 
about educational policy at 
inherent in thirmind of 
to the propagatfor’ of. 
have noted this {n-remge places in the ‘hal East Africa, 


ry best zeal we can muster in . 
these proposals, 
and sometimes raucous debate 


an to give great concern not only . 


for instance, where phe. dverdge groge national prodygt per . 
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race, but also to itq education. 1 re, ‘ 
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inflvidual is $14 a car, $id year per individual— have® 
‘noted this concern {n, thel rattitude toward. education, In the 
éducational system valluble; ¢ | them, the parents must put 
up the equivalent o five: poh 8, which is about $14; One 
wonders what a f doe or subsistence since, on the 
average, the whole part of the national’ income accruing to 
the individual is represented in educational expenditure. 

‘1 have visited crude schopls: mud and wattle huts coyered, . 


‘ with thatched palms, no doors, no windows, no floors, simply 


rutic benches. and a teacher standing before the class with 
a dog-vared book in his hand with forty or fifty African chil- 
- dren sitting there-with thei# hands quictly folded in their laps. 
and their eyes glued on the teacher (no wiggling in these 
-‘schools). During every such visit, F have had ‘the feeling that 
if we could have the measyre of commitment to education in 
our country that they make so munifest, we could move ahead — 
very rapidly -indced. 

Let me inject another note jast to show you what this dedi- - 
cation amounts to: | talked with‘a teacher of what they call 
the fqurth form in that country, a very intelligent, capable 
Aftjcarf. | said to him, “What one thing do you think would 
give you the greatest help in’ carrying on your work?”, 

I thought perhaps he might say a mimeograph machine 

yi projector or some of the other things that one would 
th nk natural in this country. ne reply was, “If | could only 
have a blaékboard!” 

This, then, is what we have to think about as we consider 
our commitment to education, Here are people ‘who in the 
last generation weré primitiye pagans and who are now cbm- ' 
mitted to education to a far greater degree, I. believe, than — 
we are asa total people. f 

My purpose, as I said at the outset, ts not to ‘give you any 
new, brilliant concuction that Has just come out of my head, 
It fs only to remind you of things that you already know, but 
which, in thé press’ of affairs, may not bo at the center of 

your thoughts. ; 
_ Everyone knows that thero, is something about change or— 
motion that always captivates, human’ attention. The child 
- amusing himself with the cmon designs in the kaleido- | 
scope, the advertising displays we see in the store windows 
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as we'walk up and down the stteets of this great city attract 
us because in them something moves, Even perhaps the eager: 
nqss with which some people go for automobile drives is an 
indication of our inherent desire to move. , 

L came across a statistic which was revealing to me in this _ 
connection two years ago when our Post Office Department - 
in. Washington reported that in the previous year 40 million 
Americans had changed their address. This may be a mani- 
festation of that old’ saw that it is cheaper to move than to 
pay rent I don't know; | don’t think ft is. | think there is a 
certain restlesyness which motivates all of us, and perhaps 
in our countty this restlessness reaches a zenith. We do like 
change and at the very same time, ladies and gentlemen, 

_ we rasist it and thig Js one of the classic paradoxes of human — 

‘behavior, ¢ | 

_ . For example, the college president:who seés that the insti- 

' » tution might profitably undertake a different coyrse of action 
had better not announce this change if he values*his hide and 

* his position. First he brings in a grup of senior deans and, 

after a casual and genial discussion of the state of the weather, 

_ the performance of the football team and what the chances © 
are for getting a little more money from. private donors Or 
the legislature, he says, “Oh, by the way, | was thinking the | 
other day about a certain mattoy and discussing it with. a 
friend and’ he suggested” (never. the president, mind you) — 

* “he suggested that we might do thus and so,” i. 
-And the matter js ‘discussed, and he says, “You might try: 
. ft out on some of the staff members and see what they think 
of it.” The, deans then go to the appropriate faculty members ° «| 
‘and, in this same ingenious way, the idea is gently introduced. 
’ Finally, the idea turns obit to have some acceptance and.” 

. merit, Then it is debated at length und perhaps fifteen, 
dighteen or twenty-four months after the president hag first 
operated by Induced initiative to plant the idea in sorfiebody 
tlse’s mind, ft comes aga demand ffom the faculty that: this 
be done, This only happens if the president is very. wise and 
fortunate, At this point, then, the change ts made and it may 

. . ‘have been a very simple“matter indeed, = “ 

_ Now. you gil reveal by your reaction to my cornments that, 

| - you understand this roulstance to, change in education. ‘Yet at 
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tho.same time—at the very same time—-ladies and gentlemen,’ : ‘ 
the pace of change in our society ts constantly accelerating. —_ 
~ Let me detail three aspects of that pace of change. The first 
fae to do. with the pace af change in science and technology. 4 ow 
( 1 am sure. that I need not dwell on these for I'am confident — "1% 
that this audience is well -aware of the impact that changes «©  .° ’ 
in transportation and communication have brought about in ‘ 
our society, We hardly think about.it when we travel at ‘a a 
speed of ton miles a minute, gix miles up. This is casual, We. | 


accept that; and yet-when we comety education, we take a , ial ee 
diametrically opposite.position and firmly , ‘hold to the position i 
4. that not a thing shall ie changed, : 


Let me recall to your, mind that our shirtent complete iv 
_acceptance of television as.a fact in’ qur socisl life today has: # 


} been brought about in just 15 years, Similarly, ibe aria 
“in nuclear technology, which has such terrifying im catignisr™ 
had its start less than a generation ago. - vant 


When the late Professor Ernest O, Lawrenceg,. then’ i ‘Xsso- 
elate Professor, first came,to the University of: California in | a 
1928, he announced what seeriitd ‘then to be the prepogtorous" cout 
notion’ that he was going to devise some means of callectiig * i 3 ae 


aa isotopes of the varlous clements: in’ discretely measurable. 4" 
. quantities. Previously, the existence of' Tpotopes: of the éleménts By t 
& ° had been detected in barid spectra, but they occurred fh such 


* ., minute amounts that hho onc conceived that, anybody. ‘could get ye 
eee ‘gnough of an isotope to welgh even a milligram: | was there. | a 
. Acgret to say that 1 participated if -luncheon discussions a 

which, with biting sarcasyn ,and ‘irohy,:thase present chided. 
Professor Lawrence on the ridicylougness of his notion: ce 
_ When he sought to offtain money to build’ the’ fixgt uccolera- Ae ‘" 

= tor, he, had oven greater difficulties, but Professor, Lawrence +." ° 
' was not to be denied, He vistted every junk Yard within fifty - 


. ' diifles of Borkoley, California, wand. from all sorts Of discarded 
ne ” machinery he wag able to ‘find in those junk yard he dssem:: 
a bled most cyrious. looking’ contraption which was the’ ‘first 


electron’ dcvelerator,” And, that va only a littl qrore than, 
‘ + thirty years ago! Today we have nut onily fairly good under- ° 
‘standing, but also application, of both the principles of atornic 
 fisglon and atdmic ‘fusion with aoe us releases, of energy. a: 
sealing theretnars: Re as 6 | 
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do they? We must seek to understand, to reorginize what we i. 
‘i know as well ay our dutlook upon it, We must seck Ways ineali, ee 
of thiy complex, and acvelerating change by which the human’: 


{ 


boing may be, the master and not the servant of technical. * 
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ae a : THs isthe pater change. 3 
a 32 AP engigoo, “the Whole electronic industry has come of. age 
a oes a Tittle mare than“; generation” In anothgy area, there is no”, , 
Rye Nr, Teds 0 detail thé“u¥mendous role in our Honomy played by 
Hg, Maen, bbe munufagturers of tl sorts*of labor-say ng equipment and 
bee ee - i®edyets for the home, | presfime that no homo:WWill bé com- y, °° 
yoos He ‘ plete iniehe. noxptew, months without an electric ¢an opener. ' 
yh One thas only ‘to vista nadel home in some new subdivision « 
rH ally: 40 sd@ at oftce, why equipping the honie of today accounts for 
a ie “yvery néarly half. gf its total cvgt. We have only to look around 
i 7 if | us, if we can’ (i a Wivthent to do so, to see quickly the 
; ! V).enorn Ae magrittude of the effect that technical advance shas & 
nt eg! bud, ofF the lives of everyone of-us. And we have adjusted to: 
re gt “those chinges}" ae 
i ve There. is no doubt either that, except for a’national emera 
Xa Keucy, this advance will continue at an ac lerated pace. If 
‘ m ity there is a national emergency, It will ardbably accelerate any- 
ns ay Way, Dut ina far different digection. Thousands of.minds are 
ee 4 busily occupled today in devising. new ways of doing old things. 
ety ', Other thousaAds of minds-—Indeed, our best minds-— probe 
e “relentlessly to extend’ the frontiers of scien¢e and technologyg - 
: - ‘frontiers which are no longerlimited to the dimensions of 
ve this earth but reach out into: interstellar, space. And again 
a we secm to be able to adjust to'that without too much trouble. We 
a: We krrow-all this if we but Stop to realize it. The pace of a 
- _ change ts far more rapid in this arcé than we generally fealize 
S * an@it ts likely to dncrease, _ a4 ; 
My question: ts this: What does’ this do to our outlook, {to i 
, our-understanding of our soci¢ty and of the kind of worldfin .) 
Which we live? . 4 oa , => | E 
» y We' cannot dgcry technical advahve,"for the mind of man’ . aly wea 4 
"4 hn its relentless quest for more:and more knowledge will not aa 
be denied. We cannot wgrn-off technidal advance as we would = |" 
*\ lturn off the water Spigot in the Kitchen sink. What ate we to, As 
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» Yet, thts pace of anes that has reailted Gi scientific and 


.’. technical change is by no means the only. change with which, 


" . we must reckon. The sheer magnitude vf present-day problems = a 


~". which society is called upon to solve requires. that new ap;: os 


proaches be devised for dealifiy with those problems. It is not 


" , _ enough that these approaches merely be new, although’ there 
bo sémo whggwould argue“in this fashion. The approaches 


ust be relevant, They. mugt take into account all of the factors _ , 


‘Involved arid they must be calculated to lead to-useful results. 


* In education, Just. in the past few ‘years, we'have heard a ; 


aa great hug and cry raised about the inadequacics of American 
- “education, with all sorts of proposals advanced to remedy 
v c alleged imitation, Fundamentally, it seemsto me that a factor 
. which we have not yet fully considered ‘has to do with, the 
_ “numbers ok nial sceking education at all levels.“Oh, yes, 
at we have all heard abput ‘the population explogion—the Hsing’ 


cafiy. qbitth’ rays, the udétl wave of étudents and all the rest—but have ,. . 
y 7 We reeo Nved that these draindtically increased and incteasing | 


; numbers really page a new aid:a different. kind gf problem,:not - 
‘ae * fimply an extension of the probjem that we knew before? - 


Pe analysis, on the critiqal mass of the materialy brought together 
* ina reaetor. Tt ig ga ifatter of enh ase that mass {8 not 
7 present, nothing happeys , If that gp present, s6mething? | 
* happens With enormous; fect. 1 suspect there | omens like 

.* -8 critical mags in. edu¢ational problems, iid 1 Was 

1, ‘ewded béttgt ‘think ‘Of the dramatically, eet ii. 
t+, whichwe milefdeal in that sense, whe 0 


ok" 7 amb confident that in. all @is, the | 
dane *~philogophiy with rpspact to education ret 
eh as gett ‘are héld ‘With convietion by. all of the’! 


mériéan"peoplg, ~ 
mt ot ipopplos:- dPequylity of educational opd@#punity, of decen: 
: Cala dion of équcationgl authority, and of humanitarign, con 


a S ‘etrn. for thet mt ual seem, to me-to. be“at-the Very core of. 
ly believe ubout education. As we'tackle: © 


\/ what qur.pgeple. r 


the massive peoblem of nyumbers,1 am-ceftain that we ‘shall be 
eo) successful onily to  Nperiesre that wo adhere emily to these 
seem pene, - a 
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Againy let usflook at some of the advances in physics. The | 
7 ‘whole business of releasing atomig-cnergy depends, in the last. 
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+ dt would be edsy tOWeVvise new ediivational ‘procedures based 

- -~" “apon ‘other princip® Indeed, we know that the Soviet Union 
-~ 2 * does just that, moe cual has rio significance‘in the Soviet . 
eat" system He is snevety a--pawn ‘of, the state, a:naked statistic ° 


o 


Fae 


Je totally lacking in human identity “Dur approach to this massive 
. Problem of:numbers@pust fe made in terms of principles we | 
_» beligve to -be right, ciples which, in'the end; are the ones 
“a | Wich every emerging nation in the world seeks to emulate, — 
a a’ ee But how-shall we do this? Does this méan that we simply. 
“8... keep on doing what we have done.in the past? Only*more so? I, 
think not, eS ew 2 ae 
+, Fam confident that we must seek new and: better ways of — 
 s7 matching the individual to the opportunity for an education 
“that will best provide means for developing his talent and his 
interest. to the maximum. The Educational Testing Service and 
other similar agencies have long Been concerned with.this very 
problem and have used all of the technical advances currently . 
availtbhe to do more effective work in this regard. Yet, there is 
& caution here, too. With the pave of technical advance having 
reached the point where from three to six thousand test papers — 
, an be scanned by a machine in an hour, one could become , 
-. » beguiled with the marvelous efficiency of such a process‘and'at ° 
-, least momentarily, forget the hopes,. the aspirations and the 
_» future of the individual human belng ‘tied up in every one of | 
‘ @hose papers. : es. & ee “gre # 
J say this ‘becausé [! myself have watched this scoring mas =: 
, chine and pthers like it; and I haye tended,to become fascinated 


‘ 


. 


oy ae 
with the flashing lights, the rapidly moving papers and the. ..... BAAN 
~whirr of machinery, There is something in.my nature’ that Just-.-/-"! 4) 
responds te that sort of stimulation; and it beéomes a genuine | 
_  cfort for me to focus my ‘mind oh the' fact that-every one of ., rf 
those papers represents one-step, and-perhaps a gyucial one, in’. *: c)) 
vidaal-boy or girls rh ‘ 


‘ 


ut cing the chat ce-for, growth of an ind) . Ps 
Saee eT “the worth of a hurnan boing TW bapdgscrihee: “ sapale . 

_. an item on an IBM card?.Of course not. [am fully convince ve 

-" that this'was never the motive nore it the motive of’ those con-*' ae 

\ cerned with the whole prdcess of testing. The true motive is jo ial 

«+ use all of the techniqués ‘available rder-to help place. the’ j; We 

Individual where he can realize hid: maximbm potential, But, "  - hi 

» dgaln, 1 say, such is the fascinatidn of the human beingtpvith,._ V7 
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_ change, nd matter of What‘ sore? that this.motive may not bes." st | 
~ understood by the very people who give thé greatest credence: - 
98 tothepresults, ©” - rc? Sg. ge? 
| | have chosen ogpcation as My illustration for the pace of * We. s. 
“. “ ohange ini focioty bécause of’ {ts natural’ relevance here today. . Ye 
~ 4,» Yet I would certainly, he.remiss did 1 not at ‘least mentton’a . oe 
a number" of ‘other. factors of social change which surely have“, - 
Wat x he on. the subjdgt. — we a 


o> 


_ "yy - fact8r Because of the rise in, the birth, rate on one‘hand and 
+ “te the advance of méfical sciénce on’ the other, our population * 
“ le today has nearly 75 per cént moré people.wnder 20 and over 65. ©. 


than twenty years ago. Thegefacts mean oat education for the a. 7 +. 
—§ ~~. young and: care for. the-aged, have assured new and greatly = 


¢ 


* splinters. ee ae ee ee ee ay ee, 
_‘ | sumit: that the only way to, solve thege problems: nif ee. ee a 
? " factorily ts to recognize. the full maghityude whic bow ia aa 
s+ have; to"gather* and gnialyze ‘Ql of thé. facts relating‘to-them:, «| * > 
* °. - gand then to seek to arrive at reasegible conckpsions): ragher ” be Re | 
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fee Ln NI, neat digress Analitagaa ce er 
het tthe wld gre dppening just abort ten. tiines faster thun'l 
3p, dvds thought posal, en. | en o — 
Just three years.ugo, | Kad thd privilege of visiting the Be glan” 74 
"Convo which \ thot thon was the most tranquil Africa} | ar 
"state Lhad séen, [-attendod several. meetings of Africans at's ¢. : 


€ 


* which The questfon of -indapendence: for ‘the’ Congo: was dis- « Pe 
. cussed. I Hetened 6 one ’speegh by: an African swho said he,” 
eg og He Mhoughe the Cong was in no hurry for ihidependgnee and if it. 
mo er pot ibn thipty-five ‘to fifty years this woyld be all right:with 
» Mthemn, That, was just throe years ago. Do you see Wherel_ yetmy °°: 
4%. factor-of ten? ‘| po ees ae ee ee, (ee 
aad ‘We now know that while’ the Congo ‘is no Tanger dell wet te 
 Belgian-control (indeed, ‘ft Is hardly undet any gontrol),.(t Zan * * 
scarcely be'said that it’has achieved responsibla independence. - ; 
“ , Now. on abrighter side, Lean report that other African coun." 9, 
, 9 “trles ®ith which | am familiar have‘ made changes in, their - 
‘ “educational systems that, are almost unbelievably in this same 
; pune Of thtee yoars. Indegd, my factor af ten stillapplies here, 
or the gdyunce has certaingy been ‘ten times faster ‘than?! 


. *rwould have thought possible: Others: who have wavéled more +” 
,’, Widely, and ‘have visited varigus 7 emerging ‘cquntries of * 


the’ wold ' tell "merthat ghey. have 
truly astonishing change) = yr 
_, Thege’eghslderations, tov, have great meaning for education... 
~ Nostonger-can. we be, satisfied With. an, cducation exclusively 9“. * 
_ oclented-¢o our traditional western Point of view. In. thfg con- 
-,  sthoctiop, Jet Me relate a yadccdote that oecurred the,otherday.’ 
_ At the Washington [ternational Genter, Which ts operated by 4: *, 
“the ‘Ameri¢ar Council on Education and where we givea one... 
"4°, Week fMitioduction to American life, L was talking with ‘anim | 
s ° * “donesian, ‘a vepy*bright young. whip with & twinkle ‘in hig eye,” "| 
' <Tuemy’ onvaMidors with him, 1 used the word: ‘orlented”4a ¢° 
: natural onough thing, a word whieh we uge alt the time, withy, \: s 
~* out being quite sure whik tt means, ©", Pg atl Tae 
At this. point, rhy young Indonasign friend looked'at mewith "4° : 
: a stile. He said, “But, sir, I don’t'wish. to become oriented, 1° * °, 
_* want to become Occidized’ a wow 7 toe 
' &! No es hag we think that the customs, the philosophy,’ e 
3 the “aBjectlves af the peoples ofthe far-cornors of the, globe +f 
from ustafe matters that can be igndred or, worse yet, cup only. | 
.o ye, o 44 woe ‘ J, 
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_ + ‘have rgeontly emorged. We in the Unite 
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be thoaght of as sph bizarre, All of those peaples are 


* now our neighbors/and what happons to them is of immediate. 


. tind pressing concern to us. In the years immediately aheads as 
is so well demonstrated in the ey ira ‘rol of the United® 


Nations, we must have: knowledg@ of these nations and we Cee 

must cultivate genuine understanding of their ways and of © |"! 

their aspirations. cae ~ 4 cae 

‘This is yet another of these massive problems of change, and ~~ at 
~ Pcould not, 1 yould not, pgaume to dffer any pat solution for: a 


“tt, But I aru.convinced that the way to starts to recognize it as 
a massive problem. ind, hen to seck all possible knowledge in 
relation to it in that context. 


: 
On this first trip [made to Africa three years ago, if you will 
permitane to become personal again, | confess ungshamedly: 
‘that every notion had abou the African peoples us | set out ee 
on my trip proved to-be wrong when I<got there, J can hardly 
remember now-what those hidtions were, so great was the im» * 


‘e» pact of reality upon me’ Yet 1 certainly thought I would.tnd the- 
. Africans Jothargic, easy-going, careless and somewhaedull. The 


facts surned.out to be axactly the cogtrary. | found them alert, 
“vigorous and decidedly demanding. ‘Phey loved to talk and they, i 
loved to‘talk far into the night. Yet they always sét ad first | 
appointment in the morning at clght o'clock, irrespective ‘of 7 
wheter I got into bed ut two, three or fou in the morping, os 
__ Now, it may- well be that | was simply ingensitive to what l -, | ( an 
should have known about Africa, If so, I confass it. Yet, the eo —= 
remains that virthally: every travelep with whom I have talked, 
Teports precisely the same experictice: a compfete change In - 
rsonal orientation to the notions that he had had before. ; te 
yis wily I a 80 vigorously for the secking of greater. Ms 
dye of all ofthe cduntries: of ‘the world, cspeotally, in 
ag of the world ip which it ae are emerging or 


States Just don’t know 
about these aread tu be able to.dome to usefully walid 


ons about our relationships to them, | a 


£0 |_- : 
“ Now ladies and gentlemen, |, have supght to ldenufy some 
» Of the? highlights of what I concetve to be thé pace of change’ 


* that results from rapidly acceleraiing developrgents in science 


~ and technology, from ‘the new magnitude ¢ tal problems 


v, a wo, . be a 
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a. * we face, and from new wegld conditions. All of these have; + 
_ _  mmodiate, dirget bearing upon’ education. 
) “In all of them, we can be assured: that the pacg ‘is rapidly 
= Increasing. We gan no, longer be. satiefied simply by doing 
/ more energetically what we, have done before.’ fh so doing, = 
, _ We are apt only to increave activity, activity which confuses c 
8 our minds, saps our energy and beclouds our perspective. ' 
“4h , ., As Tsee it, pur great need today {4 to come to.a fuller 
» realization of Mis pace of change. No matter what the'8ceming 
demands of the moment,.io matter how frantically we trey, 
” 4 ~ to keep up with the distracting dengdnds that the complexitics 
of modern socicty put ypon ud, we must recognize that it { 
the height of futility to confuse activity with achievement. 


This, is the muaning of the pace of change 
>. 
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THE THEME. for this afternoon ig “Tostings in’ the Language "7" 
Acts.” | recognize that the phrase “language arts” may’ have 


‘sour taste for seme, The Council on Basic Education, appears 


to feel that this phrase is a nice example of “pedagueso,” and 

that courses in the teaching of “language, arts” are intellectual 

sumps of a low order, about on the level of driver training , 

wnd ‘safety cducation, will grant that some things which 

pass as “layguage arts” have hardly anythihg to-do with lan- 

guage and are far from artistic. Nevertheless, | find this is a 

useful and inno¢uous phrase for referring collectively to such os 

things"as courses In English language and |itcrature, foreign 

lahyuages- and literatures, and yos, even pyblic speaking, oral 

interpretation and forensics, Tho phrase “language arts” com- 

nits us ta no special pokit of view, in my mind at least. 
This afternoon's, theme brings us to'an emphatically inter- 


‘discfplinary part ‘of ‘the conference-—not that there are any 


part(s of education which are not interdisciplipary (and I some. j 
timés think that its interdiscipliniry nature 18 at once* the, 


‘ virtue and the, weakness of the enterprise of education), But.‘ 


this afternoon wé-are reaching out disciplines other than . 


_ psychology and. statistics. ‘The firsttwo of our speakers are, 
| by profession and training, in the field of the Hberal arte; Th | 


thett work in ¢cducational measurement, they ure likely to be 
gast.in the role of what test makers sometimes call “subject 
matter expéttn” Let us remind ourselves of the respect we’ 
must agcord fo ‘the subject prattor@kpert—he must be an 
oqual partner —-indeed, primud inter. pares, .“first among || 


— gquals—-in- the raat tests, It happons, however, that) 


our speakers fromm the ‘libera) atts this-afternvon can be even 


"| “more equat,” becadse both of them have had sufficient export, 
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ence in test making to have become much more ‘thoroughly oa 
*, acquainted with testing problems than many, I suspect, of 
their colleagues. - oo, ; e . 
‘The first speaker will:be Dr.aHarold C, Ma 
General Education A at Harvard, that is, 
may Eriglish .composition, Dr. Martin dn | 
comparative litrature and has, been at cacher of .. 
English-and of French, and later a“high 
fore he became assaciated with Harvard tly*he was 
appointed tg be Chairman of the College Entrance Examina- 
| on Board Commission oh English, established to help in ~ 
“ Improving the tgaching of English In our secondary schools, * >>’ 
~-His topic is “Testing for Elogance.” oS 


- Our second speaker, Dr. Wilmarth HH. Starr, formerly of the 
University gf Maine and currently with the Modern Language ° 
Association of America before he takes up his dutics at New, 
York University next year, will introduce to us, some new ; 
developments In the flel® of foreign language testing. It is in 
* . connection with his ¥uk that We win pay our respects to the © 
National Defense Edjication Act, as [ mentioned this morning, 
since one of the prgy{sions of that Aet is for studies of methods 
and, materials in the teaching of foreign languages, Dr. S arr. 
iy the director of a p uject to develop hew tests of competency 
for Wachers of foreign languages; this project ‘Is being con- 
d by the Modern Language Assocation under contract 
with the Languaye Development Section df the U. $, Agfice 
of Kidugation, When he is -not working on projects like this, 
he ts a professor of French lunguage and literature. His topic © 
4s “Compotency First: New ‘Tests in Foreign Languages.” 
Our third spoaker will be an olf ‘friend to many of us, in. 
fatt a Sarmer ip ot te of these" Invitational sConferences, 4 


, Director of. 
c in Fresh- 


4 


Dr. Irving Lory¢ of ‘Teavhers Collage, Columbia. Dr. horge: 
has spent mdd/of- his professional life working on proflems 
"of teaching the language arts; the Lorge ,readabigty formula, | 
_ the Lorge semantic” count, the Lorge-Thorhdike: Intelligence 
Test,.and a number-of other things have made Lorge ‘alniost 
a houschold word among educational mecnusurement specialists * 
= and cducators' generally. Ho will spdak. ubout “Estimating 
Structure In Prose”, 225 | 
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| WE ALL KNOW that ihe {s: widespread discohtent among lay- 
& men at even the best Meyts 86 far: devised for measuring stu-. 
dents’ ability to. write English prose. That discontent forces 
reconsideration on any. who are concerned to, protect the ingeg- 
~rity-of measurengent ‘as well as the wolfare of students, Sich 
recongideration yocs on constantly among testing experts, as 
} am well aware, and it may be impeytinent for a layman to 
presume in a field so‘recondite. Yet [ am moved to-do so 
, because I- believe that a puristic a rdach.:to’ the “problems 
such testing raises docs morg harfn than, good at this time 
and because, ih/at least one instgnce, the lay voice has pre: 
-vailed with a résult that seems to me intrinsically bgd. - 
- dee My formal acquaintance with testing iy limited to’ seven 
" years on the-Committce of Examiners in, English of the Col- 
lege Board and to legs extensive work on the: Advanced Place: 
. ment Examinations of the Board .in thelr catly years, Now, 
eo the English Composition Tost: and.the’ Advanced Placement . 
' Examinations in literature @nd composition may not be they .. © ; 
best so far devised, but they havé at least attempted something” 4: a 
beyond the measurement of rudimentary, skills (capit@lization,, . 
‘+. punctuation, spelling) and of clomentaty: information (com. 
". Myon usagg, parts of. speech, identification of dyntuctteFetruc- «: 
i Wares and “go op). Because the English Composition Test is 0s.) 
a designed for ns ‘“ administrations: ee quick corregs ty," 
tlot. and repor 


<a 


. 8 


oe 


has, ‘as you probably know, ‘developed’... 


1. "} oblique procedures,’ It. purports, for. Instéiace, to measure 'the "|" ’"). 
Pia 8 we td ‘organizo a paragfaph or a complete essa by requit- (ake Bye 

ing = 

> op chronolo 

a. @structurd 


‘topi¢al, 
al order, ‘or to craate frém undifferentifted tems, "::" 
eee setgeare and: —— barter ot 
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he es tg,put.scrambled santenced, dn lo 
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~ > - On otcrsion it has used one original line of a quatrain along 
mi with three spurious lines to measure the examinee’s sense 
of rheaning, his ability to diMinguish appropriate from ine | 
‘Appropriate diction, and his car for linguistic rhythms, Some- 
what more directly, it has set him tasks of re-writing sullied. 
prose as a way of testing his capacity to compose felicitous ° 
‘gq  ptose of hisown, oF. _ ES 

_ J trante here“ onfy ‘the three; most successful devices ‘tried: 
se In the Bnyltish Composition ‘Test. My files shaw at least a dqzen 
feather Pdovices pro-tested: with’ varying amounts of success *.. 
noe but sgldom with ‘chough to encourage use in a‘arge-scale \' 
admifistration, ge 8 7 


>|. ; adh yt ; 7 
Thpse approachés used and re-used in recen years on the, | 
‘test, Have had reasonable success as ‘success if measured inf ‘ 
such matters. ‘That ts. they have shown themselves to be— 
( ether taken alone oF in conjunction with other tests-—-deéently 
satisfactory predictors of the ability to write such prose as the 
college freshman year demands Insofar as grades at the end 
of the freshman year are themselves pellable ingasures of 
_ competence. To be precise, the: ECT is less: sudggssful than 
she Scholastic Aptitude ‘Test’—-Verhallas a predictor but, taken 
| . in conjunction with thg SAT,V, is Mare successful than. either, 
soy ot the two uged atone, %~ 4 | 


rot 


" "Perhaps I should qualify that definition of “success” to allow. . 
stind. in the analysis yf results ofa multl-section test it is. 
customary to congratulate oncself if cach part score shows a | 


“om approved as: evidence that} “Uoviating part ts succtasful ..*. 


* confess that the aBparent incongruity» makes:me uneasy, —... 


S. qute /agdinst it has grown rather than diminished ifthe | 


* 


a va direct, meusuyrgment of wal ah 
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Joy for @ statistical subtlety Fhaye so fur not been able t6 under. + 


a Strong similarity to ‘the whold score; that I think, is called OU 
eee high reliability of parts. On the other hand, I have heard sharp, 
J deviation of one: ’eore from gimier scores on the game test , , 


‘9's because it measures something different from what the other 
ie parts measure, Phere’ may” be more compatibility in the two ie ae 
ae > positions than TY aware of; ingly present ignorance. must, Y 


a “Depplte the sucecss, however defihed, of the ECT cat gthe a 


ae *” past toh. yeyurs.’ Last -yagr it found: vent ina move to gubstitute 0: 


. i Marold Maltin ~ ae 
most obvious procodure of all, that of requiring students to 4 
write an essay on an assigned topic. ’ : ae 
“ ‘The attractive simplicity and gommon sense of such 4 pro: \ \\ 7 
‘cedure proved, when the chips were down, more powerful; eo 
than any arguments that could be.brought against ‘it, and ye 
“that is not to be wondered at. But tho new test—or “writing af 
sample” as it is now ‘euphemistically, called—is, I believe, . * \:; 
inherently defective «and will itself have to yield after trial to’ : Ay 
a more realistic facing of the central problems that beset all i 
__ efforts to test skill in composition above the most rudimentary - v 
“+ " "devel, Yet even if—dg when —the “writing sample” shows its 
faults in its. initial fOthcoming trial, we should not, in“my ' His 
judgment, be content simply to-return to the old‘forms of the, — 
‘test with a feeling that the failure of this experimen fully i 
justifies its' predecessor. |, oe 
The dissatisfaction with oblique and objective testing has: 
its source in al genuirtd- oncern for the processes of ¢duca-. \s 
‘tion, a matter often. ost sight of by those ontircly. committed i |, 
to the subtleties of. measurement. [t is, moreover, supported 
by the frustrations and fears of thousands, even tens of thou- 
~ sands, of. people for whorn*the Bourd tests’ now often seem 
to constitute a power of decision @trageously disproportionate 
to their actual value. If there is any reasonable and safe way , 
“+ of coming fo teyms with:that concern for. the processes of 
ic education and with the worries of countless parents and ‘chil « . 
dren, we ought! to try it aven at some cost tor ‘goals ‘which Sieg Be & 
testing experts have in large medsyrd considered inviolate. 2 
Permit me to preface my proposal ‘with the tentative criti- 
cism that testers may have honored the relative where ae 
should have honored the absolute and may have sdcrific 
much for the absolute where they might well have srt 
themsees and their customers with the relative, Let me com- ‘ , . 4 
pound the wqystification in this synenyim : they have been satis) | 
» fied to acccfg what poople do rather than what they should — 
‘do but unwillipy to let them be what they are. 
aaeee™  Tepeak, of Bourse, about language, not about morality, and a 


‘the first part of my conukdrum refers to the practiée of accept 
ing: the, cammaon lecution : nda satisfactory one, a practice 
vs “encouraged by the deliberations, tf not always by the dogmaa, 
te fe miyotcen Wngulots, Inyny ae most of the error—for 
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such I think itsto be--in' this matter is oné of omission rather 


than of cofmission. No test-maker has yet, so far as IT know, .. | 


uccepted “ain't” or “aren't 1?” as a correct response to a tosting 
problem. Most have been wary even of allowing such disputed. 
locutions as “due to” without a nominal referent or “hadn't 


rf condaned by the simple process of evasion. : 
,  Thé rationale for omitting items for wh ch the common. 
rol locution is neither efegant ‘nor precise from a sophisticated 
fest _in'the use of language, or ignoring the results if they get 


Le _ Tf nearly all-the oxaminecs ‘respond with a. common byt in- 
offective, locution or; in. sorhe situations where the error {s 


presented for correction, fail to respond ‘at all,. the item is". 


\ _ deleted Or not scored since, to’ use the statisticlan’s phrase, 
‘ _ It diminishes ‘the validity of the score, Such items, in one. 
ms ‘analyst's terms, are “wasted.’* Not for ¢ minute would | sug- 
| :/ yesh that there ts among testers a conspiracy to establish the 
oo! \ language of the unedueated.as a norm or even to hasten the 

“' processds of Ilrguistic change. All the testers | have known 
'aye—-in spd@ich, at least—Iliterate persons and fis fastidious 


hae as the English teacher about linguistic convention’. It is nones _ ‘ 


Aedes theless true that.the net result favors latitude rather than 
rigor, what is rather than what should’be. 


The present English Composition Test, for example, very ° 


rarcly carries an item requiring cerrect use of the apostrophe, : 
~~ either in’ possessives or in contractions! it has entirely -pro- 
> “scribed the distinctions’ between “that” and “which” in relative 


a " Festrictive- expressions. It ignores misspelling$in matters whigh | 
‘{*#. Frise above: clementary conventidns, like tha 


clauses and tho distinctive punetuation of gp na and non: | 


2 r@@eriness is indigenous to the state of American usage. 


a tttudinarian as it has comp to be in its dogma, the test is 
. often decried as unreagometfy difficult when, that is, it is not 


ought” or “most perfect.” But the lapses have nonetheless been ~ 


« Into a test by accident is, | am sure, statistically unassailable. \: 


e of pardllel con: *, 

tf .@trucgin, inversion, precision in the choiee -of adversatives, . 
. * gund gho like, it is often forced by the statistical’ rationale to” 

ra forms it can condone only by the bleak admigsion that . 


» s « being decried as unreagondbly casy. The simple statistical fact 


6, " Analysie of ECT Data by ETS staff, Nov. 30, 1957, i, 
a . Se eg 
ant ae a: ~% 4 


‘ Urry, ; to 4 ae 


ra ae 90 e | se ® 
’ \s 7 - . 
' 
\ ” 


sot 


* . 


IC MeN ots . vo Jha , . yi . Be 
: F ao Wiser is y : F 7 ie ae . o . 24 i cle 
2 ee @. é ¥ r) ' 
= «|. 8 a ‘ : 
i YY ° ' 


. ta Harold Ge Martin” 
a that the fayther west of ‘the eae you go, the fower the 
~. . average-score becomes may ‘be’ ag coefficient of the fact that — - 
the farther west you go, the smaller number of. examinees you 
find’ taking the test, ‘but I am ihclinéd to doubt it. I suspect 
that, instead, we have here some ammunition, for those who, | 
charge that these mass tests, even in their broail-church form, 
_ contain unconscious biases, that they arc, more than we admit, 
_ instruments designed to play the tunes characteristic of the 
. metropolitan East and therefore clicit only caterwauling or 
~ &llonee from those to whom their frets and valves are alien. 
_ The matter is no joking one simply because it is.as hard to: 
establish the grounds. of an absolute demand in language as 
: It ts to determine what-a relative standard is relative td. 
: _ “Where's it at?” may be a.respectable ‘locution in Indiana, bit’ 
in Massachusetts it. js unmistakably a vulgarism. The more, 
subtle you try to make a test of language, ‘the niore you attend’ 
* to diction and syntax, the more you find culturally validéted © 


ee easier _and the-more differences you find, the more you .. 
ure'forced to widen the gauge or to ellminate the measurement 
altogether. . 


The problem,.as 1 see it, is*that testers’ are(forced, by their A ‘ 
. determination to make one test do for all, to ca themselves. 
The more they attempt.to measure the ent population on ube & 3 
one scale, the more they must limit themsolves in what they *. : 
test’ The result is bound to be as ungatigfactory to those. whos ot 
ae hold in reverence the ‘linguistic’ conventions Inherited’ from 

‘qeventeénth- and cighteenth:century English plain style (the: 

Dat) as to, those for whom the plain style is a eek 


= A - almost as much importance as ‘those locyti mis mined from the 

GP a wat lode (the MiddlajVest and West). % 

i If it is ‘true -that dictiOh.and syntax vary’ sighificantly. from’ . 

“region to region’ in this country---the syntax of the Midwest 

1. nd West being t more Jaconic. thay that of the East, the diction - ote Sa 

su’ + of the South more periphrastic, openly metaphorical, and — 
Latinate than that of East or West—then test-makers’ who oe 
want ty deal With anything that might properly .ba.called “So "7" 
“elegance” In-style fage“a cholce between plumnping for. =a i re 
to the. disadvantage of those committed to: another 4 evialig - i 
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° ; .. . My preference iss for, the first choice—that js, provided that. 7 


_ the one plumped for is thé style which | find m pleasing. 
Judging trom my reaction to the. current enforced Yatitude on’ -* 
fome items of the ECT, I doubt that | would cheer so heartily 
for the election of. Southern-fried diction, Is the rhatter, then, 
purely a relative.one? At the risk of rousing indignation, | 
must say that I think it is not. df the language we want to 
establish as the norm for educated ‘people in the United States i 
Is that bust suited to rational discourse, | believe that the 
language of the Kast has an edge on that of the other sections _ ~~ 
of the country. It'may be less colorful; it may be less “natural,” 
‘ whatever that means; but it y more predise, more strictly. 
ordered, more amenable to. analytic proeedures, 
If 1 qualify this claim, as I do, by admitting that books of 
' prose discourse written in the West, Midwebt and South are 
similar in style te those written in thé East, I shall have to 
compound my offense—or offensivencss—by obServing. that 
the style for such bédks ts largely set in the East. New York 
City dominates American publishing, and thé cditors-of the =! 
— large publishing houses temper-the manuscripts they read by 
: ~ «Gastern tuning-forks. But our concern’ is not really with the 
stylgsof fature people why write books, only with that of 
young’ people who write examinations. And, in fact, we are: 
not so-much concerned with their free writing as with the 
kinds of problems jo which they cari respond on a test’ 
woe os Short-of creating a national academy, like that in France, 
“>. for regulating offi€lalase of the language or, in the fashion 
_ é., odor nineteenth-century England, a Society for the Purification ~' 
8. Sof the American Language in the “Pradition of the Eastern... lines 
ts States, J see no feasible way of making my parochial view. ° 
. Of stylistic truth prevail. In theory, it prevails today, of course, *' 
sind that ts exactly the trouble, as far as our tests are con- 
e@ cemed. The expectation of a particular standard governs the 
Construction of the tests, but the demand for. performance 3. "4. 
. .  constsicnt with that standard is modified ‘and subitantlally . 


« 


! 


Ma. 


ae 


» » °  weakeried by the exigencies of statistical calculation, 
. In the face of this problem, I see no alternative, if” there 
“is to be testing of prose style in 


utters beyond basic ‘con: 


, ventions, to the second choice-—thatfof devising some mans! ° ). ; 
a - Of admitting the authority of the vafious natlonal patterns of 
7 pagese raed 


Te 


ea iy al volt. ‘Martin 
‘language. I may be wrong about this matter, but it seems to ° 
. me that the problom is nog solvable on fa national test, My 
. - proposal therefore entails more than'ong:test and-more:-than - 
‘ one center of reading. And by “more than ‘one. centgt: of” 
. ' roading” I meat to imp sotiethiig other than @ mere: ‘aie. = 
“8 - . tribution of redding centers to which the law Lam a Pringoton ~ . 

, __ ,Slanai goes forth bysmail dr telety : 
What | should like to see tricd ue neuen tests of‘com-- 
parable difficulty but: ‘differing inscontent offered ‘throughout wo 
the country; the’ administration io. he followed at ongd. by . a 
‘large number of" reading sessions held in the: ‘center of diny o 
_tegion populouY enpugh to furnish from five ito ten thousand °° 
papers, For each a chief examiner, working with other exami’ 

_ hess of the area, would set the answers and:the: standards to 
- "be applied. Atlanta Elsie’ paper would thus fall into the hands « 
_¥ pdople- fof whom hed maniner Js as familar as breakfast. 
ominy, and ‘Boston Jolinny’s into those of-readers, lee preter 
their optn-uncontantinated by lye or soda water. . «+ 3 
~ VA: further. benefit that might; derive frém such. deoenit 
|. Hon would ‘affect the charactey of the’ test itself. In the a 7 
SOS place, it would make possible’ the retention of nyany..items —, 
Na _ now deleted, whether or not they produced: a‘high incidence’ . 
“af response, simply because f would be reasonable .to Assume | 


' 
Se oan 


oy °« that the. failure to “tespend indicated a failiire to learn the 

Ld, aecepted: ai it Of a “regio.” Thé assumption, of eourse, 

[ .would be—as It must be—-that the’ teaching was, consistent: , 

with the syntax.and idiom of the Fogions, and ldcalized inade- i. 

. - *quacies in performance would bey. a fairly sound indicator of 
_ poof instruction. ‘The feedback frgm test to. curriculum might - a ie 

‘then become more definite than ‘it -now Iga dondition [8S te 

_ pellevé to be cminently “desirable déspite*the ifuet that teasers | 

‘are almost paranoid jn aie feat of being ugcused of dictating, ” 

courses of study. - » 8) oe 3 

cw ‘In the. second plaeé, it might.beoome pisatble, ylven ‘(let . 
. “us. syy) thirty reading genters instead of one, to reintroduce 
by, oo sume: direct writing to ‘the vr in the form not. pf complete | 
iyo. espitys, but of eee resporises to carefully contrived prob . 
ders, Here, of course, we run into the fambbar bugaboo of | 

. “dll essay testing—the known unreltability of essay #coring-and 4! 

7 UN: intrinsic unreliability of va ingle ae as Seen, or a. ae age 


, 


i? 


*” “The objéctiofy to’ such ad ptan for introducing greater rigor. 
) fading -of-papers: wloWliig that groufl to deflne rr os 
dts standards are many, It would be more costly, of course, 
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“ane, 


or four problems rather than a single snd inthe testing period, - 
‘a practice followed by the Board. for a short! time. during 
World «War IL: Repeated attacks ‘on writing problogs,- even 


within a brief period of testing-timte; offer a better inti ation © 5 
ofa student's compositional competence thari’ a single ex- 
tendedyéssay,, And. if is my éxperligee that single paragraphs . 
. can be.read-with greater reliabillty’ than a’ many-paragraphed -: 
essay. Even more important, hyweyér, ‘is the-opportunity a. _ 
, large number-of. reading ‘centers offers for establishing among‘: 
*‘the readers a degree of-agreomeitt abdutsstandards, semiething: = 8 4 


"manifestly émpossible “without ‘rigorous and continudd ‘disgi- 


pling if, the number gf readers in a group gets much higher | 
* » ee al : et 


“than den or fifteen, ° 


In a given 


-but T refuse to consider that a serious objection since it is 


entirely uneconomical to economize in a testing’ procedure 


on which the expenditure-of. an ong dolar or two for each 
candidate is insignificant in comparison with thetatal expendi- 


ture “for educatign which the) tests help to initjate. It would = 
baradministratively Onerpus; but if it ‘iss not administratively, 


“impossible, I sée no ob ction In that: what are administra- 


tors for if nowto carry burdens, cach administrator a kind of 


pons asinorum, 9b to speak? Finally, ft would crejite multiplé 


Standards of excellence instead ofa single ong. ‘That, quite * 


: in’ writing. In some measure, byth 
af difficulties can be' diminished by: setting the Gandidate threc . 


ve 


clearly, is the abjection on which. ‘the entire proposal must * > 


stand of fall, | mo : 
. The sanctity of a single stundard, as I suggested ¢ 


arlier, {8 


“more, than ‘a fittle sullied> by’ thie compremises that must be» 
“made to ensuro It."For while ng-one today can claim that a 
score of 662 mears one thing abguna candidate from a rural - 


i Rocky Mountain school and another ‘about a candidate from 


Phillips Exetor Academy, there 4s more than a suspicion’ that, | a 
ane score docs not mean cnough about ejthter. What: would, - : ie 
happen If 2Q0- 


800 yeale for one’ tegion .were markedly 


“different from the'¥00-800 scale in angther? For nincty per- 


cent of the colle 
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-Narthaast, [-thirtk ft would make sno difference. at all, simply 
begause that ninety percent will be attending: colleges in the 

‘ region of residence. And for that ninety percent the discrimi- 
nation possible once the whole scale was confined to it should, 
"| believe, be considerably more valuable to students and to 
college admissions, officers than it now is. What, then; of the 
ton pércent who yo"outside the region and af. the diaspora of 
Northeasterners? [ sce fio more objection to a formula of 
adjustment whereby the scores from‘ one region ate scaled 
down and of another raised, preliminary. to comparison of 
candidates, than’ ¢® the present uniform scale which, ‘as a 
recent article in Life magazine shows, consighs institutions 
to categories of real or imagined quality on, the basiy of thelr 
median scores. g ns 7 oe 
' Wo are atl sufficiently indoctrinated. 4n this country with 
the scientife concepts of virtye to admire procedurey that are - 
simple and clean-cut and to stlspect those that seern untidy 
and varying in-their controls. But we dare ot sacrifice all for, 


‘tidiness aad uniformity jin a procedure which has a kind of | 


* magical power over curriculum and human hopes. 
_ Last spring 1 talked with’ the admissions tutor of one 

. of the-volleges at Oxford, a distinguished literary ggholar who 
manages to run the admissions job with just .a io clerical 
help, His college -seleets one out of Agye applioghts. ‘The 
examination papers, of course, are read by gthers, in a leisurely 
fashion, over, the summer bpreceding the fall term, and the 
choseéf) applicants ave notified gf thelr clection carly :in Sep: 
tember, | asked the gentleman if that procedure were not 


hard on the eight candidptos rejected out of every ning. He ° 
¥ B } 


brushed my query adide with the remark that Americans 


—, believe in equality, Englishmen if Itberty. 


A 


Thé comparison {8 scaretly a newsowe, and it occurred to 
ine at the time that the liberty’ he spoke: of seemed to be 
rather more useful, to the college than to the candidate: one’ 

i, had the. Hberty to choose garefully and at leisure, the other 
_ the liberty to go hang if bé Were not chosen.But there {8 bite 
in thé remark al] the same, Equality we want, even in 80 


~~ s amall a matter as testing for competénce In composition, qut 


equality ts a rofative matterf not a. universal one. We. may 
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rot As, opportunity for developnygiit; put it Js entirely possible to | 
: “y _ 


legislate equality and produc 


ee of justice.And, 
‘further, it is possible to be s¢ 


‘onitnitted to a uniform equality 
» , that justiflable local differengys are: penalized, That, | believe, 
is an inevitable consequencd'of any nation-wide test that'tries 
to discrimihate those who ‘write well from these who write 
poorly, when “well” and “poorly” refer to an accomplishment , 
| - . " greater than. simple torrectness IR the“use of isolated linguistic 
rd > Items, ee _ 
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ANATOLE. FRANCE Is reported to have said that,the world would , oa 
ceme to an end from a glut of-réading. 1 .think ‘that jf he had 
beert here this late Ootober Afternoon, he would have been , 
~ sure that Armageddon. was close ‘at hand, but he woyld have 
revised his statement to read that the world would come, to 
an cnd through a glut of testing. | assure you that Ido mot 6. 5 7! . 
mean*that this thought would put on the sidé of the angels i oS 
your speaker who has just presided over the design and de- , 
velopment of 62 tests of seven competencics, in five languages, 
and their administration t6 well over 30,000 casés, geographic. +. \- 
ally distributed ftom ,Copenhayen ‘to Guadaldjara in the 
summer. just past, But how did thig come, dbout? <y0-s 
1 ask this question because it gives nig, tHe comfort of resort!" - 
ing to my professional habit of ‘never getting into a subject: . 
without submitting an historical preamble. Yet it seems to rnd: | 
that it-is exactly such an histgrical, perspevtive that we''must 
all keep before us when we ‘would addres’ ourselves to any... ° - 
question related to foroign languages. I am sure that you dre te as 
all aware ‘of the impact that World, Wat I had upod, the, 
aducational system-—-p&rticularly that phage‘of impact which\4 , "j, 
brought the-rdalization’ to America that, whether we would’ | | 
havo it-40 or not, oup accumulated global éspaQsibilities have ,, : 
placed us in a position’ where the ability to communicate with an 
other peoplés in thejr vwn tongue and with understanding ay? 
of their vajucs as scen from within hjs assumed a ‘primary = * 
importance. | am“sure, too, that [need not remind this'iddl- + e 
ence that this sudden realigation had a profound effeot on, oad 
the teaching of foreign languages in our-country, Tl need’no | 
more than romind you that until World War II the typical ': 
pavern of foreign language cducation th our country whee ~ 
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two-year sequence of a foreign language, generally taught by.” 
_, people who could not speak the‘languaye they wéde teaching. 
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- Furthermore, they taught the modern foreign language as.they’. . 


o? 


4 ; 


themselves had been taught--in the classical ‘tradition’ of, 
grammatical analysis “and with emphdsis upon translation * ~ 
‘from the foreign language to English. Unfortunately this ‘is. 
sstill the pattern in too many schools, or should ] say-in too 
schools since less than fifty per cent oo public. , 


few 
SCCU 


ndary schools even offer a foreign langurs : 


But an fmpgrtant and exciting erm 


steady development of’ new attitudes toward the-teaeching of 
_ modern forcign Yahguages in-dur country since the war, This." - 


“quict revolution."pas it hiya be 
language teachers, hag'{been characterjzed by fguk new 
‘emphases which | would describe .as follows: 


I 


A new brdcring ‘of and stress upon the language: 


* skills*—i.e. listening comprehension, ‘speaking, read~ 
ing, and writing; . m 


on) 


_ The inclusion of knowlcdgeability about descriptive’ 


lingiistics (not anew discipline in itself, but one 
new to the general. foreign language teacher) in {ts 


. applied aspects, This concern for applied linguis- . 


cs is based on the assumption that insights: into 
the nature’ of language are: of advantage to the 
teacher in that they cnible him: to predict: q.) the 


difficulties the student who has alféady acquired - 


often called ampori? farelgn 


‘the habit patterns of English’ speech, will have th. 


ny 


encountering a new sound.system an@b.’)) the diffi- 


culties this same. student will have iit contrasting 
the new sound system*with the now graphic system 


of the: foreign language he is learning; Ae 
he ’ ; . ° 
‘The growing conviction that cultural insights {nto 


whose: lanfnne being Igarned, have ‘qual: tm- 
hn | 


portance withthe language skills; that fn dealing 


_oyWith such cultdral insights, tha.eqntdm porary:aspéct 


~ tudes Have primacy over ‘th 
i ‘ a ’ : P y bs . 


A. 


has primadéy’ over the historical, san and’ jttl- 


‘ é 
‘ wy 
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‘Whoa lenis and behavioral patterns.of the peoples , 


¢ more traditional ‘vata. . 
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- “Joging of monuments and artifacts; valucs are of 
. more immediate concern than discussion about the 
quaint and the picturesque; , |” “ 

4, The conviction that the, teaching” of foreign lan- 
guages is a profession. with a methodology that is 
: subject to scientific analysis, that the methodology - 
can be taught to the novice teacher and that it can 
-# be demonstrated to have beneficial fesults in terms ~S 
of the ,abjectives implied by the foregoiyg three . ~*~ . 
arguments. 


4 


~A second dimension of historical perspective must also be 
brought, to bear upon the question before wus. It is the 3 Foes 
and growing dissatisfaction with the pattern of American 
education which tends to‘qualify teachers’ quantitatively under 
the assufmption, woich has been honestly held, 1 am sure, that 
hours of contaétwith subject ‘matter in the classroom, and 
‘eredits gained’ thereby, would insure the necessary qualifica- oe 
. tion to tech. But this pattern is valid only if it rests upon a 9 
| Beerantee that the hours of coverage are adequate for maptery ~ 
of the subject to be taught, and thdt the credits carned have- 
4 valuc that implies some degree of ‘¢xcellence, | would not 
“here raise the question of Whether this theory applies equally 
to all subjects, but 1 would point out the discrepancy b&tween 
the theory and the feality “as‘it applics to my own diseipline. 
* The nature of foreign language learning is such that it involves 
in the mastery of the listening, speaking, reading and writing 
objectives skill learning as differentiated from-subject matter, 
coverage. Since not all huniadn beings are able to.naster skills 
4 at the same rate-or to the same degree of ckcglerice, tt would +, 
‘gecm to‘me, as“it has. to many others, that measurement in a 
. these’ areas should. endvavor to designate the degrée ,of . . oa 
~ competency acguired, and that such competency cannot be - 
‘+ measured quantitatively but must of necessity be measured 
' qualitatively, = ° 4 oe a a 
a It ls important at ‘this me to keop 4n mind that if the, s dpe 


skills of reading and writing lone werg involved, only new, 

directions would. have to by udded'to tnedhonored measuring | .. 
. Inbtruments such us those familiar to you all under the label ...7 * 
__ of Cooperative ‘Tests, Regents Exhminatiohs, gr College Board © 6. de 


ee page 101 
i ia os Py ot 
4p ee, 


‘ee wee oe . j a _ va 


ae 4 


. 7 
’ : . y) 4 
i 


4 
A 


960 Invitational -Conterence. On THating Problems” 0 Ba : a 
nS ae : a, 
as Px. (examinations in foreign languages. But you will ‘remember 
| - . that in appraising the chariges that Have beeraking place 
in foreign language learning I implied that the direct heritage 
of the methods of teaching the classical: languages, the 
‘grammar-translation approach, has been generally discredited / 
in view of the fact-that it did not produce people able to com- 
“,  municate with other peoples in their own language, Although" 
eo. * the tests mentioned above have been somewhat affected by 
* the new efiphases in language learning as I have been describ- 
ing them, they still have a tendency to emphasize lexical items _ 
as contrasted with manipulative. skill, ‘grammar knowledge as, 
contrasted with automatic habit patterns, and translation 
abilities as contrasted with fluency in the language. - 


4 


° 


Even if we “assume, as dots-scem.to be the case, that the. ° 
* "above mentioned testing instruments can be expected to... 
undergo the. changes implied by the new emphases on lan:. - ™ 
@ 20° guage learning, the two skills of listening comprehension and | 
f speaking -are still inadequately acounted for and I must. re- , 
| mind yYouragain that in the new order of events these particular - 
oo: skills Have assumed enormous proportions, I hasten to say” 
that I amr aware of, and ‘that I welcome, the fact that the 
» College Board examinations ‘and the Cuoperative Tests-now |, 
* have available optional listening comprehension tests for some. 
. languages, and that there ls. some indication that speaking 
x tests may be In evidence before tap ong. People as familiar . 
“a8 you are, however, with’ test. developméat.are aware of the 
of long time required for the development ‘of, new test forms 
’ x rn 


om the.above channels,‘ oh - SS 
: ' : : , — 2 ae «8 : . . i 
‘0, * A Third perypectiv®’ of an historical nature «now nee sto. 1, 


be added to the above ‘two, It is the-extrendoly rapid, rather. ° @ 
_  heterogencous development of’ ihe application of: electronic... 

devices to language learning. Only the first few ‘chapters of 

si thiy particular story can Ie read y'daty;: but’ the phenomenon ; 
- known as the language. laboratory (and at-this’ pomt I cannot . Bee 

“refrain from saying one that may be fendered obsolete nearly =: 

any day by tho perfection pf the teaching machine) has:added | 

a yew dimangion to language tcakaing: Acquisitig of. audio.‘ 

lingual skills, which in the past tended to be incidentally. and 

canuutly taught, hag buen :constdegably reflned-by, the potential 
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“that the’ language laboratory. has for training, through Bys- 
tematic imitation. and drill, those skills held: to be so vital. to 
, modern foreign lariguage learning. =”. ; 


Now~the language laboratory, although it has seoutciod us 
“with new devices for testing and measurement, in the case 
"of listening comprehension and speaking skills, has also added 
: ae a new conctrn: that of the aisiderable expenses involved 
~, in the production: and: sooriny ,8f tapés and in their successful 

use as measurement deyices for which, large’ scale, reasonably 
". sensitive! and desigably standardized electronic equipynent is 
seguir It is probg@ble therefore that we could not have antici- - 
pated adequate batteries of tests in these two skills, nor, indeed 
new tests in the others, that-syould be designed to measure 
competency and that would be tailored to meet the require- 
ments of the new approaches to foreign ‘language learning, 
_ had it not been fo the possibility of tapping the monetaqy | 
_ tesources provided, by the government funds and the wide 
scale, human resources available. from insjde the profession 
‘once ‘such funds were at hand. . , . 


* * case: that the atmosphere +s ufrrounding language learning 
J has notably changed ih the last ten years or, so; that a new 
emphasis. has beet stimulated in terms ‘of competency 
* — measurement, and that the application :of electronic’ devices 
‘to lapguage learning have provided us with new instruments 

for training and theteby for measurément. ww \ 


I shduld ndw like to describe to you two large scale testing 


_ declared purposes competency testing in/strict accord with 
al the implications of the forégoing statements: about the 


~ 0 "1955, as on of its -many ‘activities, theeSteering Gommittee’. 
“4 of the Foredgn. Language, Program of the Modern Language 
. Association of America: published the Qualifications for Sec- 

i ondary’ raed Teachers of Modern Foreign Languagos—a 
i ee aay which was dpbsequently éndetied by “eighteen 
—_ natignal hnel regional, organizations of lan teachers. The 
. \, statemen gta ished three general lévels ‘AB cioncy mini 
+ ) mal good ‘an a sal for seven — of language tener 
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May | assume at this point ae 1 have made the following 


| programs whith are in progress, both of oe have as their | 


a urgencics and emiphases on forolgn language learning. In 
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competencies 1) -fistening. comprehension*, 2) speaking, 3). 

reading, 4) writing, 5) applicd linguistics*; 6) culture, and. 

7) professional preparation. a eo 

mT February, of 1956, at a conference of natjonal leaders 
we 4 represéGttng: the field of educational administration, - the need — 
“ys for deve opment of standardized proficiency tests as an aspect | 
‘of teacher preparation ‘and certificatior was endorsed. Obvi- 

ously a statement of desiderata in relation 

¥ competencies, no matter how strongly rgpresentative of a con- 

' sensus of the prafession, would not be as- effective as the 


g, the program has been” 
dgdicated to the ‘principle that, these: tests would be produced /’ 
‘by the profession and “for the profession. Over 200 language . 
teachers, most of whom are experieneed ‘in test degign and: 
developthent, have been involved in the production of pre- 
liminary test forms in the seven competencies and in the five’ ° 
languages, Freneh, Spanish, Italian, Russian and Gerrgan, It 
Roo ts these tests that were administered last summer to well over - - 
Me 2,000 teachers in pre- ahd post-test situations Involving two 
; ~ forms of! each test. ‘ SF es ac 
‘oe ‘It is too early to predict the final format of the tests, but 

( submit that the prel nary: versions are both imaginative 

and exciting: The list ng comprehénsipn and speaking tests 
. tre produced on tapes. Spoken by native, speakers they deal 
3 awith y variety of Situations and dialogues which orfe could: . 
“ _¢xpect’ to encounter i the foreign.country. The speaking tests ‘' 


“Involve reading aloud to tugt pronunciation, stress, accent, ‘|... 

: _» und intonation, a section whith calls for imitation of. a model," _ 
an a section to tesymantpulative skills iri earns of changing'given 
: ce » Originally designated tig aural understanding and language analysis 
‘<r ae ‘respectively, - ae ty ‘ fete “ 
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vegetal patterns, and situations pictarially presented which = oe ia 


": inyolve manipulative skill and fluency | in. Tes ponse. ‘The‘tead~. | os 
« ing tésts are aoe measure sensidiyity: ta: style, ‘ghades ° i 
P 


“ the native language and those to he acquired: in the foreign — 


7 


, examinee botk-to hate principles of linguistics ; as a‘factor ° 


‘at measufing the examinct’s abijity to control the methods' @ @ © .- 
_ most likely to produce competenc 


- of the-MLA Foreign: ‘Language Progyam Research Center,and 


of meaning, and com 


hension of the message. The writing * > ea 
tests: inculcate variou? 


vices ‘such. as reconstructing: para~" i eke 7 


graphs, ‘interlinéar corre tion of “error, ‘completion of tek: ”_. oo . 
graphic messages, and structured compositions, The “dpplied ©. “24 
linguistics tests endeavor*to rfredsure the compotericy of the oo. 


of his own skills, anif-to show his ability to to Maps thoge prin: °. 
‘ciples to predicting areas of, learning difficulty ‘that are the; wa 
inevitable result? of’ the conflict between agguired habits: int” vans 


language. Thé culture tests endeavor to measure competency. 
in dealing with behavior, as well as interpreting literary and a : 
artistic production.',The professional preparation ‘tests aim - aoe 


dn the stx’ precegling ‘arcas.. 
A.“second large program,is currently being initiated, “also” 

under Modern Language Association direction, which will tesg -" |. °! 

agaln in five languagessthe skill competencies bf the haat 

‘learger from grades,7 through 14; ° . 3 
Directors of the classroom testing project are Donald Walsh ~ ashe BS 


a’ F ¢ 


Prafessor Nelson Brooks’ of Yale. ‘In announcing the contract, oy 
the United Staths Office -of Edueatlon Stated that “traditional 


“methods of teaching forcign languages’ in the United ‘States 


hitve emphasized the teading and writing skills, The néw tegts “0 
of Speaking gnd hearing dre expected to havga- powerful effect: Mew 
of forcign language teaching since they‘ will make it much | 


_ easier for the teacher to emphasize ‘all the four skills in keep- amy 


ing with the intent‘ of’ the National Defense Educatign Abt” Oe, 


Thus youwill see that the above statement reflects what I have ” a, ; 
‘heen sg@ying: previously abeut the realignment of emphasis ey 
_ toward the audio-lingual ‘skills. | " an a 

Together” the two. sets of tests will thus presume to mivasure Oe 
the corhpetenc|es: through ‘all levels of language learning and 
teaching { from grade seven upward. It is appropriate to mention 
to,-this audience that both of the’test batteries are being de- 
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. -Edycatiqnal’ Testing Service'where Mrs.-Miriali ‘Bryan ‘is now’. 

| ' ‘serving as the EFS Director of the MEA programs, lt is,{a0 early, po 
1. + “OF course to predict the révults butit is the hope arnidjindeed the |- ., 
Pt)" *" conviction of all those, conceaned with the. projects that these "4. 2 
ag ‘testing Agtivitigs herald a new4tend-for testing‘in the subject + = 
oO ondtter’ sciplines, one. that-will be characterized by the pro-  ~ e 
| .. ‘duction 6£"nation-wide norms: and staridatds’ df competence ~ \°f 
"+ within the discipline throughout the whole spectrury of learn. , *... 
‘y° 4%, Jing upward .from the elementary grades 4nd including ‘the., i 
“0 7 teacher... a ae ee | 

» .+ >; Ansindication of -general professional intetest*in the’ pros’ ' —-/ 
"oe gram is provided in the following watation, from the annual, : 

- , repgrt of. the 1959 meetings of ‘the Counc lon Coqperation, in . 
eg Teacher’ Education :|“The Council on Cooperation.in Teacher a 
. ~ Education recommends that rescarth, and.development in. a 
~* + proficiency examinations 0§ all kinds-be erigouraged’ 4nd sub- |‘. 7 
i « ___ Sidized! The Modern ‘ L: age Association .in; prdposing © \'.” 
> . requirements and in devalopiny tests for assessing. competency + _ '; 
. ‘in, the teaching, of ‘foreign, finguages: hag set an example . | + 
i . that may be followed by -otly academic ‘and professional. | 
oe 4 ‘diséiplines,” roe: ee a a i ae 
Soh bam net ode. to predict ‘the’ rsults yee and design = «et 
og s prévioys to the dhe when ‘the data is-fA and evaluated, but ' 
ane v fam appy to offer this diseusdion to you .as alk of _: ie: a 
7, , “how members ofa subject matter discipline, when armed with ~~" 
el _ “the necessary resources; ‘can exercise thely respohs 
ee oe - clarify professional, standards by producing ingtrumem§ ‘to 
Ate ane measure their own teaching competence, and those to meas N 
wd oo ae compétencies their students, acquire in the times in which 
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_ Tedchers Worl-Bodk (1), published ip 1921, provided a mor 


te appropriate grades." Instead of. judging thie ‘ditficylty of the ° 


* seotion off. the Stanford-Achtevement Test, indic 
each book read and liked in the, year’s supplem¢ntaty reading.” 
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SINCE the 
obtain or to develop a valid, reliable and impartial method | 
fa ing the comprehenstbility of textbooks. and related. . 


8 . 
nses of pupils on examinagons based on tho texts they - 


/ have read, on ratings and judgments of the understandability. ” 


for children by teachers, librarians,and professors, and on the: ' 
expressed reactions of children to books. E. L. ‘Thorndikes . - 


objective’ basis for estimating the difficulty of tex¢s‘in t 
of relative frequency (or rarity) of:the wordsugedy / 7: 
The concept of readability, however, received.{its' primary: 
impetus from*the researches of Vogel'and Washburpie “an | 
‘They werdasked to assign. books for supplementary reading “« 


a 


books, they based ‘the Classification’ of each book’ orf the tested 
About 37,000 children, who Had-taken the paragraph@ndaning - 
on a ballot - 


SS ee , on Pe : . 
Note: It is a pleasure for me to- express my appre ation to the staf. 
members of the Institute of. Psychologidal Research, Teaphers College, 
Columbia University, who made most vf the tortvots gnalysep: Miss 


.-’ Mabel Wilcox, Mrs. Dorothy Heft, afd Mrs. Lugia lap atrge Gteinzor; 
to my former graduate studénta, Dr. Joshua Fishman 


d oh Walter /’ ; 


*MacGinitie; to thé College Entrance Examination’ Board ard Educa- 
tlonal Testing Servi¢e for supplying the candidites’ Genera}: Compost- ‘ 


tions; (6 Dr, Jolin Carroll ‘for supplying thg 150 passages froar his 
of students» 


in Freshman Eriglsh in the Sch@@ of General, ™ , Columbia 
Univiraity; for Meir cooperation, gS Figg yo ES 
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turn of the century, schoolmen have beep trying Pa. ‘ 


1aterials, These attemptg have considered the re- . [ im 


reading ability.of children who had read and‘ likgd the book. © + 
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_ It should°not - aia therefore, that grade placement 


* density, and human interest of a text is titvartant for any or 
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For each of the 700 titles which had’ received at least 25 | 
favorable votes, the book's score was the media of the read- 
. Ing grade scores of the children who had read and liked the 
book. Thus, the ‘Winnetka list (3) provided an empirically 
graded list of 700 supplementary books, _- 


But what of now books? Or of books that had been read 


. by fewer than 25 children? Vogel and’ Washburne tried: “SS | 
+ Identify factors within the book that. would predict’ its grade - 


placement. Thoy were the first to'dev. op a multiple-regression 
‘formula predicting: grade placement from four factors internal. ] 
to the book, namely, the number of different words per thou-’ 
sand words, the number of uncomtnon. words per thousand. © 
words, the number of simple sentences in 75 successive sen-"" * 
tencgs, and the number of prepositions: per thousand words. 
The Vogel and Washburne Grade ‘Placement Formula (4) 
became the prototype for a sorics of multiple-regression equa- 
‘ons which predicted a criterion from the internal aspects of . 
the book-~c.g.,. vocabylary load, sentence complexity,. idea 


of 
f 


_-densyy and human iaterest. 


"No other integnal eténient of a text added significantly: to’ 


estimatec hade placement, or difficulty, In general, in most 7 
~availabl multiple-regressiog oquatiohs,: such ag. Lorgé (5), 
Flesch (6) Dale-Chall (7), grade placement is estimated from 
‘the weightéd cop tribution of af least two of the fpur elements, 
estimated {fom Vocabular load, sentence complexity, idea 
rearrangement of the -séntences in a toxt. The. Lorge Reading 
Grade Index for the Gettysburg Address, for example, would _ 
be the same no, matter in what order the sentences are pre- 
sented to the unwary tay a we uo at 
No readability, formula ‘credits structure, or organization, 
or coherence, or sequence, or unity, While.tt may be assumed 


* that nearly all@ublished books have sume structure: or organi- 


zation br unity, few teachers of English compusition would: 
grant that structure is an inevitable ingredient of student. - 


In thé search for some valid and equitable basis'‘for evaluat: 7 


_ ing candidates’ contpositions and essays, the College Entrance 


. 
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page “e  - 
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"-tople, “Pres$ures of Modern Society’ that Tend. to Diret jthe |" 


~ words relevant ’to the ‘assigned bfoad’ topic. The, candidat 
a was instructed to spend five minutes in giving the theate in. , 


a 


ability Index As: basdd on/a count of: the: numffer of words * 
written, the number vf ser tences, the number. gff prepositions, + 
_ and the'number of different. hard .words. Tha four counts * 0 ~” 
“are the basis. for the valyes: average senteney! Mgth, the pro, 
. portion of -prepositional phrases, and . the . i ‘tion of dif. ae 
- ferent hard words. The réaders’ ratings ‘for ¢ seis ri “2 
! for Mechantes, Style; Organization, ,Reaso An Content, Ae 
as Well'as the candidate’s Scholastic Aptitude i Verbal and | dhe 
ae Matlernatical aptitude scorés, wore supp oH METS. oY 


_ candidate's compositions were related to structure’ Efpq agted -’ { 


_leving Large: hs 


"Examination Board and the - Educational Festing - Service 5 
Indeed, -have “devoted . consjderable research endeavor. For 
example, ‘in May’ 1956, candfiates for the CEB -Achievemeht .". °°: 
Test in General Coufpssitign (8) wyre asked to write on the.” * +" . 


Lives of Young People. AS a, basis for their éssays, the ve WEE, 
askod first to read a’ relatively ‘casy passage’ of about 1900. . J 


one or two’ concise sentences, another 20 minutes Jn prepar- 
“Ing an outline, gn§'75 mihutes in writing and revising the 
casay. 4he candidates were, informed that the essay was’ to de 
be judged for five qualities ! Mechanics, Style, Organjzation, | 
Reasoning and Content. ' 

‘Since. two of the quulities to-be vonsidéred in avaly hing the .. 


the Educational Testing Service: to send mo a sampt {lof the ° 
compositions. The purpose was to ascertain whetherjstructure 

or style, as rated by the expert readers -of the congppsitions, =» 
might be related to the Lorge Readabillty Index¢¢ for the. 
compositions. ETS sent the 194 compositions whic&/had been - * 
used in- eee the reliabilities of the readerf ‘Ni had Oo, 
evaluated the Genctal Composition. From ‘these random ws 
sample of 69 vomnpod tlohs was scleged. For ‘ef K of these,. ; 
a Lorge Readability Index’ was computed.. The) it brge Read- e. 


Since an IBM 650 was: ayailable, the caf nue matrix of 
- intercorrelations among the 24 variables in inputed, fol- ~ 
» lowed by a factor analysis and fotation/b the ser's va imax, - 


(9) of thd five qualities by two readors, th 
~ length, ratio of prepositions cage th 
and the Sf T V and : scorn 


erage sentence © Nw 
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* ‘Phe ‘two qualitits of major ee the estimation ‘of 

structure were'the ratings. for’organization and style, Accord: 

+, “ing to the directions to the readgrs, “Organization had to do*' 
cP with the extent to which the writer has formulated a clear|} 
.inténtion which lent order ‘to his essay and had, as a con$e:’ 
+ + quencg; arranged the blocks of material in a reasonable and 
} effective way." Transition’ between paragaaphs (or blocks off- 


" material) was considered under organizgtion, but tra itiol 
4, ‘°° ) within paragraphs was consideted a matter of syte fa Doge 
). Style was to-inclade “choice ef words, construction o a, - 
2%) ¢ fences, and the internal. organization of ‘paragraphs . . ie 
vo is, “. . . internal construction, of the transition of the payh- | 


graph had to do with transition between sentence elements 
+ €nd the general flow of language within a paragraph unit f’..” 
; The first orthogonal factor was identified as general tica- +: 
; demic ability; the second, as a specific for the first reader's 2: 
ratings on organization, reasoning and content; and the third, . 
a specific factor for the.fdea density (in terms-of prépositjbnal -- 
« phases) and word choic® (in terms of differént hard wards) . 
in the composition, ; i ~~ 
— ‘On the ‘first orthogonal factor! academic ability, the’load- 
dings for Mechanics, Style, Organization,.Reasoning and Con-  ‘ 
tent by. the first reader were .80, .83,. 78, 72° and .70 *' 
respectively; for the second reader, the ‘lo; dings rgspectively 
were .83, .86, ‘81, 81 and .80. Or the same first-factor, the 
loading on sa) Verbal score Was .77 and on the SAT- 


e 
t 


Mathgmatics score\:72. On the‘elements of the Lorgé Read: 
_ ability Index, the loadings on the first factor were 04 fay . 
_ ‘ “average sentence length, .34 for ratio of prepositional: phrases 
\": and .32 for ratio of hard words. rr ua 
“a - * Apparently, the readers cannot evaluate organization-inde- 
a :., pendently of the “halo” cither of the general goodneps of the . 
object-—i.c., the composition, or of the person——-t.e., th 
‘date, The clements in the Lorge Readability Index, , 
oe seem to be measuring aspects of the composition whic 
gat ' yelated more to. the composition than to the bre » More | i 
- t., " + gver, the Lorge Readability Index, as may. have bee ‘bxpected 
fs on logteal grourids, {g related) in a minor way'to r atings for 
". . dither organization gr style, A sort of “haloed” general evalua- 
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tive judgment is subsumed in the readers’ ratings for the five - 
| +” qualities. ts 4 se .@*% = 3 
... Arlother opportunity to attempt to evaluate structure and _ -. 
-.. \\ umity was affoyded through John Carroll's massive and sig- - 

nificant study’ of litérary style. (11) Dr. Cagroll collected, a 
sample of -150 passages from, various -sources toerepresent 2- 

very wide, range of content and style. Each of these passages 

was judged in, terms. of “semantic differential” adjectival scales’ 

. and’ independently appraised in terms of objectivé measures. 

. The average of the ratings given by cight expert judges for 
each cémposition on cach of 29 semantic differential. adjec; i 
tival scales and the actua} gounts for each passage in terms 
of 39 objective measures—e.g., number of paragraphs, of 
_ sentences, of unmodified nouns, provided. 68, écores for the: . 
matrix,of intyrcorrelations a Je ac 


r ” 


On factor-analysis, the matrix was resolved as six orthoganal - 
factors (or at least nearly orthogonal). These factors were 
identified ds goad-bad, persofal-impersonafornamented-plain, 
abstract-concretg, scrious-humorous, and descriptive-narrative. - 
¥. As must be’ recégnized, this represents an important résolution , 

of the 68 scotes of literary performancesinto six independent , 
components. 7 . ee ge 


For my interest in appraising structure, attention Was 
‘directed to threc of the “semantic differgntial” adjectival ' 
| scales: namely, those*for rating the passage on a‘seven-point . _ 
' scale for meanlnglessfneaningful, hazy-clear, and chaotic. , st, 
ordered. Of. all the experts’ judgments for each passage, these *  —« .* 
‘three seemed to'me tobe related mest nearly to what intuitively ~~, 
ay be considered structure. 7 rn ee <2 
‘These: three adjectival judgment scales” hayg loadings om 
Carroll's: first factor, good-bad, of .61 for meaningful, of 63° * 
for clear, and: of .58 for ordered, but apparently are@inde- . 
pendent of any of the five other orthogonal factors {dentified, ee! 
_- by Carroll. None -of the objective scored (whjch are similar og. * 
‘fo those used as>clements in readability formulas) had anf ea 
significant: loading ‘on the» first fector. ‘Apparently,eexpert 
. judgey using “semantic differenttal” adjectival scales dtilia@t «0 ¢ - 
a sort of “gentral goodness” as an tnternalized referent for. |”. 
_ judgirig the passages ‘for gnodt of the 2G aspects, 
, , “Ppage 115 aos 
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It seemed useful to ascertain the-oxtent to which the Carroll ‘ 
La orthogonal factors may be. related to the +Lorge Readabilfty Pil 


1. Index. De. Carroll gencrously provided the 150 passages and 


the &ix factor scores for each passage. For each of, these 150; oe od 
» Passages, except, the one that was an-incomplete 300-worg 3 oh 


sentence from ja -will, the -Lorge. Readability Index » was 


. computed.” : . 6.2. ue a 
ye A Kaiser Varimax rotation’ yielded at least two orthogonal © 
Ney ‘ Pa 7 7 , ‘ ) 
ne factors: thé: first, defined by Carro|l’s ornumented-glain hac 
i *. loadings on average sentence length of .42, on xatio'of prepo- 
_ Sitional phrases of .44,' and on ratio of hard words of .41; 
: suggesting that Carroll's ‘namented-pjain ‘summarizes what- Wace d 
‘4 ‘ over elements are measured inthe Lorge Readability Index. 


- The second factor, identified by Carroll’s personal-impersonal 

factor, Boe related to Lorge’s ratio-of prepositional 

» Phrases"witl’ a loading A -.65 and the ‘ratio of hard. words 

‘\  Nelth a loading of --.40/ Apparently, structure’ or unity was 

: not identified in Carroll's factors nor in the elemayts of the 
Lorge' Readability ‘Index. mo .. « 

Is structure, or organization, then, so integral a part of a 


_ it alwayssubsuined in a halo of “gencral-goodness” or general 
, ovaluative st? piles structure ‘or’ organization of a text is 
ae “ taken so much for -grdnted that only exceptional deviations 


In organizati¢h are recognized. Some of you may remember . 
youy first difficulties with. the strycture of Dos Passos, or } 
» @ertrude. Stein, or James Joyce. Perhaps some of ‘you may. *. 


remember Bartlett's (12) Remembering, in which he. asked 

+ ~ — .subjects to reproduce the Indian talg called tle “War of the 
‘Ghosts.” Undoubtedly, some of the memorial inadequacies: in 

. Feproducing the original text ate related to the Indian ‘folk 


tale’s unusual structure for Hritish or American ‘subjects who’ 


have interhallzed a “more ldgical” structure, Sequence, unity 
_ and coherence may. be sd, well learned that experts accept 
“, most written material as stryotured. Of course*handbooks on 


a4 “writing refer t the ‘necessity for “good” structure in English . 


prose, ‘Textbook discussions about goud structure imply that 
' “structure,” “coherence”: and “good organization” are practi: 


y eglly fynonymous. Words’ such as “unity,” “ordor,” “luctdity,” 


“ 
» ie eo 


ow... aid “sequenée,” tf not used synonymously, suggest at least a 
a ae is Ret > OU | | 
: ‘ i: : TF 

ae ee Lty | of 


+ written passdye that it cannot be evaluated or measured? Is , 
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close oe with | I cones ‘of “good” sthcture ha, 
one .of these terms;"As- yet, has. been-defin adequately. 


Some _ understanding: ‘of thal range of concepts implied by | 
structyre can be had from the ‘contexts in, which the words’ - 


are used: 
Handbooks abut writing usually do -definc “coherence,” 
“organization,” unity,” and “sequence.” In defining coherence, 
_ for example; Greever and Jones (13) indicate that ‘ ‘cohcrence 
‘implies a close and natural sequence of parts, where every 
part of atsentence must have a clear and fratural connection 


with the adjoining part, A sentence is coherent when one idea | 


is. .completed at‘a time, and when the idea which naturally 
comes first in ‘thought or sequence is ‘placed first in the-sen- 


+ tence.” For Goodwin and Guill (14), cohékence “is the natural 7 


and logical order_of development of a sentence, paragraph of 
, composition, tlrdugh a, single idea. To be coherent, words, 
. phtases and clauscs, must. stand as near as possible tS the 


_+ words they modify. The development of the-main theme should 


‘follow a‘ natural and logical ordgr. Succeeding paragraphs 
should be an amplification of the’ topic paragraph.” Baldwin 
(15) considers coherence to be synonymous with order, that 


‘is, “a composition is coherent when the people that listen to. 


it follow it readily, and when cach part pie for the next, 


* To be cgherent, a composition must take hold of both subject | 


ve matter and audicncé. It should be agranged according to a 
- plan; since natural thought sequences are not orderly.” Perry 
“Hb 16) indicates that coherence occurs when parts that belong 
together are pliced tagcther, ‘Writing is ‘incoherent when in- 
& qf saying all that ore has to on ono topic in one “place, 
ore writes a little, changes the subject, remembers something 


vf thought rather than mere cofsecutivencss, us 
w Organization for Burrows, Fereber, Jackson and’ Saiinders 
(17) is. “clarification from the verbalization of ideas,” although 


and a i involves ed releyent levels 


» 


: . " . 
ane oa ; 19. ‘ 
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they state that “the power of organization ‘is not yearned ‘but 
: follows ‘need’ in.a concrete setting. It develops with the clear- *. 
‘ness ‘of purpose Jor which specific material is being gathered 
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°. that. might have been | sald gn the first topic, says it, and after . _ 
saying it, resumes the second topie.’ Perry emphasizes that. 
“cohcrenge demands unity in. the sense of orderly conneeion 
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_ in tegms‘of” a specific probiem, A feeling for organization ‘is 
:: © an*indate part of everyone.”. ; <a " 


e Avdefing Greever And Jones, unity’is “a combination of = 


related idéas where each gentence contains only one thought.” 
Baldwin calls unity “& principle of clearness involving .expan- 


sion of an idea and the fixing of interest,” and Goodwin and’ 
¢ Guill state that “unity is present when a theme presents a 


complete whole and holds firmly tq a point of view. To secure 


a well-organized whole, a definite plan must be used. The 
point of view should not shift within a sentence, and: the 


‘beginning of a paragraph should bear strongly upon the central 
Soe at . 


idea.” 


Burrows, -Fcreber, Jackson and Saunders - indicate that 
sequence “is the proper ‘bundling of material,’” and that “the 


ability to use sequential writing develops. in children matur- ° ~ 


ationally with the dawning of a ‘sense of order.’” 

These definitions, however, do not give the principle of 
structure nor suggestions for many procedures’ needed to 
achieve ppoper unity, ‘coherence, or sequence in prose. . 
' Thespasic problem, then, was: to discover some operational 


~ demonstration of unity of prosc.. If unity means that sentences 
follow cach other in an orderly fashion, then if‘any passage | 


were to ‘be fragmented info portions and put into random 


t 


order, cduaated’ subjects should be able to reconstitute the | - 


portions into a useful order. If this random portion procedure. 


were adopted, then two kinds of questions may be asked: 
(1) are people agreed as to the structuring of prose passages, 
and (2) if such agreément is-found, is the reordering an aspect 


- of unity, coherence, and structure? ~ =. sh, 
____ For investigation of ‘the operational approach, three stories. 
from Reader's Digest of October 1954 and four passages from 


Kucore of July 1944 were.selected: Each of; the ‘three’ stories 
from, Reader's Digest was divided into twelve or thirteen por- 


tiong ee 150 words cach, The passage entitled’: 
an”. was chosen because it presented several: short: 

_ thenics (marsh birds, Captain ‘Frank’s life, Louisiana: iris) 

, loosely connectéd with each gther, without any -logical or 


“Nature 


chronological order. The passage on “World Bank” was selected 


“ 1 a8 presgnting’two major thomes (Eugenie Black's efforts for. 
“c., thd World Bank ‘and: the character. of the bank itself) in a- 
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ica sedee i c., topiceATcading ta tople B, back to topic A, 
then again td, topic: B, etc. The pieeeycalled “Russian Story” 
was chosen because it presented one consecutive theme (Lt. 
Cqnel Grigori Burlutski's activities,in the Russian Army). ' 
~ The four passages from Encore were each divided into their 
teri ‘to thirteen’ cpnstituent sentences.”“Some Old’ Families” . 
presentedetwo major themes (description of Catamount, West 
Virginia and~“Bill”); “The Assasgination of President Garfield” . 
presentod several aspects of Garfield's life ¢ freemasonry, scrap- 
‘ book hobby, mastery. of. language and science); “On W. §. . 
Gilbert” presented two major’themes (people in Gilbert's - 
serious plays versus those in Gilbert's fairyland); “The Last 
of R. S. raprciented one continuous theme, (the last illness of 
R.S.). : 
' * The pstiions teach ‘story atid the sentences of each pas- 
. sage were randomized and numbered. A disarranged story of 
' *dissocted portions, .and a disarrariged passage of sentences 
were. cotlated in each-of the poggible combinations of two at 
a timg, The, gets were “Jistrbeees ‘a homework assignment 
to students in freshnian English classes (Al, A2)'in the School - 
 .of General. Studies .at Columbia University. Each student; thus,. 
was’ giver.”a set of scrambled portions which he was to re- 
. arrange tgto'a “complete story, and’ 4 set of scrambled sentences 


for arrangoment into a paragraph. Students were asked todo ~, 


the assignment independently, and: to reeord what they judged 


. to be the correct ordgr by"writing. the numbers of the portions — 


‘and of the sentences on‘a special answer sheet. 

Frogn the returned responses, a random sample was selected. 
‘Vor each vf the three’ stories and four-passages, the first 48° 
papers ‘from the yandomized pile were ‘analyzed... . 
_"¢,, The. abjectives were: first, to ascertain: whethef these sub- 
* “eats. would: agrec with. one another ‘In’ the: ordering ‘of ‘the 
disarranged texts; and second, to‘ascertain how many breaks 
_ _4n sequence «there, would be. For estimating” agreement in , 
 “ordax,, Kendall's: covfliggent’ of concordgrice was used, For estl.:, 
niating recognition of sequence, the nusnber of breaks from « 

publis ed séquence was obtained. 

» Intuitively, for the passages with disarrangtd, portions the 
expecigd d order from most structured to least was. expectad + - 
. to ber’ "Russian mony. hate Man’ and “World Bank.” The: ° “h 
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“ter oe Subjects ‘agréed ith the ranks, given thé disarranged portions’ , 
We awed termy” of Ken (18).:coetficient of concordance ag '70, . ae 
oe" 86 and 51. whteh put the ‘passage considered least stric.: ~.° 
mm * “tured in the last position: but’ did “not, put the other’ passapés ©. 
_{n, the “expected” order. But, on ‘the average: nuniber of dé-) 
* partures ‘from-sequence, “the expected was: found—i.e.,:2.4;° | A 
| 2.7 and ©.5, or in percentage terms about 16%, 20% sind 43%. °° 
moos departure from ‘sequence... * te , BS Fe a a 
© 5 “For the disarranged sentences. the intultively. judged order . 
. _ pty £0r-gtructure Was expected to be: “Some Old Familics,” “The * 
' Last of R. S),” "President Garfield,” -and “W. S,'Gijbert,” The, - 
agreement in assigned ranks did got follow the “expected” 
‘order’: the coefficients of cogcprdayEiigyerc' .72, -37, .52; and 
".37.. The average departures fron): equence, ,however, ‘were 4 
‘more nearly in line with oxpectations, The averages were 3.4, a 
we 5.6, 8.4 and 9.3, and in percentage terms 21%, 36%, 46%, - \ 
" and 62%, If these: two leads are suggestive, some “departure : 
from sequence may ‘é used as a measure of structure, | a 
>: *. Dr. Joshua Fishman and I used still another approach for 
se estimating structure. For the same disarranged portions of 
. the stories, graduate students itt Teachers.College were asked °: 
-* to firid for each portion the other portion that goes ‘best with | - 
if cither forward or backward. ‘The results placed*the three 
- Stories correctly in the intuitive ordef bf Structuredness—f.e,, . 
@ “Russian Story,” “Nifure Man”’ and “World Bank.” . ae 
On the bases’ of : these several’ explor tions, it seems that 
“Structure can be ovalualid independently of “gencral good- _ 
. Hess” or “evaluative” estimations. The ‘arrangement.-test ‘which _ 
“is souseful for establishing the struggre in a set of pictures - 
6 ae ee of cartophs seems: to. have: utility i Bene at the structure’ “« 7 
oo Bees” tn prose: The departures’ from published sequence, on the de: . 
oe < ~ parturos' from an order ace ted by’ consensus may serve ‘as 


wy 
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the basis of-a new approach for judging structure—ulthough’ 
[ doubt thatsit is an-approach-that’ will be relished by oither 
| “4” teachers of composition - or by expert: réaders. oe 
panememandins? I peneral, subjeots found it aster’ to put portions: into 
SS eae “proper order than to rearyange scnitences into proper sequence; 
Thiy undoubtedly reflocts the redundancy of tore informution. - 
glyen within a portion, Convergely, it*is more diffighlt to put ° 
s i a 
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: sentences ‘into ructural sequence * because of the | pr 
~ “enough redundancy. ‘ 


The ypethods, though not new, may be applied. tb measure Oe 

the connectedness of the: text materials. Indeed,,the measure 

.y-may,be an addition to the ratings. by expert Judgnient of . a we, 

hae ae and organization in. written prose: | are J 

‘Pegh&ps .a partial solution for the, problems of rating ort a: 
evaluating ‘printed materials-and candidates’ compositions is: 


to get rid of the excess in syngnymy by’ ‘restricting the evalua 2. *  %: 
ve words to two: (1y coherence, for sequence from -serttence” oo ee 
‘to, sentence and from paragraph to paragraph, aiid (2) | sy: 
-organization ‘for clacity in the dopelopment and expansion of =... 
an en > ‘ oe ib 
a. ee 
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